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CHAPTER I 



BACKGROUND AND DEVELOPMENT OF THE PROBLEM 

The library catalog, acknowledged as an indispensable 
index to the library's collection, has been subjected to 
severe criticism. To the library administrator, the catalog 
represents the end product of extensive utilization of li- 
brary funds, time and personnel. It is, therefore, under- 
standable that much concern has been expressed regarding the 
efficient preparation and maintenance of the library's col- 
lection. 

In response to this concern there have been numerous 
catalog cost as well as time and motion studies which have at- 
tempted to determine how much of the library's efforts are 
used in such preparation.^ While many of these studies have 
dealt with the problems of the more traditional card catalog, 
recent efforts have also been concerned with the economics of 
preparing and maintaining computer-produced catalogs. Al- 
though such endeavors are desirable, the decision as to format, 
content, depth of analysis, etc., must still be based upon an 
understanding of the functions of the catalog and an evalua- 
tion of its success in achieving them. 

^For a recent summary of such studies see Donald Duane 
Hendricks, "Comparative Costs of Book Processing in a Process- 
ing Center and in Five Individual Libraries." (Unpublished 
Ph.D. dissertation. University of Illinois, 1966), pp. 23-35. 
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Paced with the choice between two methods of catalog 

preparation and maintenance, the judgment as to efficiency 

(a cost factor) can legitimately be made in regard to systems 

of equal effectiveness (a function factor) . This basic 

problem was aptly stated by Reichmann when he reasoned that 

"no evaluation of operational costs is realistic and meaning— 

2 

ful unless the final product is taken into account." 

The indexing of library holdings has undergone many 
changes throughout the history of libraries. The catalog has 
appeared in printed sheets, printed books and on cards. The 
card catalog was introduced in the United States during the 
mid-nineteenth century and "two or three decades more were 

3 

to elapse before it was generally accepted." Since that 
time it has become the dominant form and exists today in 
arrangements known as dictionary, divided or classified. 

The term "dictionary" is misleading in some respects 
but it has come to mean, traditionally, a catalog that ar- 
ranges all types of entries into an alphabetical sequence — 
with certain chronological and arbitrary exceptions (such as 
the rules for filing subordinate agencies, books of the Bible, 
an author's collected works before or after single works, 
etc.). The physical separation of the various components of 
the dictionary catalog into separate indexes is known as a 

^Felix Reichmann, "Costs of Cataloging," Library Trends , 
2 (October 1953), 313. 

^ James Ranz, "The History of the Printed Book Catalogue 
in the United States." (Unpublished Ph.D. dissertation. 
University of Illinois, 1960) , p. 1. 
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divided catalog. The most frequent separation is the two- 
sectioned catalog composed of an author-title sequence and a 
subject sequence; other combinations are possible and do 
exist. The third type, the classified catalog, "is neces- 
sarily limited to subjects, and in it the entries are ar- 
ranged according to some preconceived scheme in which related 

II 4 

subjects are brought together or associated with the other.' 



Dividing the Catalog as a Met hod for Improving 

5 

With the exception of Fletcher's article in 1905, the 
merits of the divided catalog have been the subject of dis- 
cussion by librarians only since 1935. While no definite 
advantage has been established for the divided catalog in 
the studies made to date "it would seem that what evidence is 
available supports division for almost any catalog of any 
size."^ Two general reasons consistently appear for deciding 
to divide the dictionary catalog. These are, in general, 

(1) the problem of congestion and/or spatial limitations, 
and (2) the complexity or illogical nature of the dictionary 

catalog. 



^ Jesse H. Shera and Margaret E. Egan, "rhe Classified 
Catalog; Basic Principles and Practices (Chicago: American 

Library Association, 1956), p. 13. 



^William I. Fletcher, "The Future of the Catalog," 
Library Journal , 30 (March 1905) , 141-44. 

^Maurice F. Tauber, "Cataloging and Classification, " 
in Ralph R. Shaw, ed.. The State of the Library Art, Vol. I, 
Part 1 (New Brunswick, K.J.: Graduate 

Service, Rutgers - The State Universxty, 1960), pp. 92-101. 
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Some librarians have stated that division will allevi" 
ate congestion at the catalog and represents a natural par- 
titioning when spatial limitations require parts of the cata- 
log to be physically separated.^ While division may truly 
be logical when space requires that new cases be located 
some distance from the existing catalog this can be con- 
sidered a temporary solution at best. When further expan- 
sion becomes necessary these libraries will be faced with the 
compounded problem of having two separate alphabetical se- 
quences to relocate. Further, the charge that congestion is 
more acute with the dictionary catalog than with the divided 
catalog appears invalid. Lubetzky has appropriately pointed 
out that "given a number of catalog users, congestion is a 
function of the number of drawers and feet of space avail- 
able" and "congestion will remain unchanged regardless of the 
arrangement and grouping of the drawers in one or several 
catalogs."® Therefore, while the problems of congestion or 
spatial limitation may serve as the impetus to consideration 
of dividing the dictionary catalog, these factors alone 
cannot justify the decision to divide. 

In 1905, William I. Fletcher posed the question of the 
capability of the dictionary catalog to cope with the problems 

^For a general discussion of the literature and the 
problems see Fred Heinritz, "Does Dividing the Catalog Relieve 
Congestion?" Library Resources and Technical Services , 8 
(Summer 1964), 310-16. 

^Seymour Lubetzky, "Crisis in the Catalog," Catalogers ' 
and Classifiers' Yearbook Number Eight (Chicago: T^erican 

Library Association, 1940) , p. 49 . 
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of the increasing size of library collections and the result- 
ing complexity of their catalogs. He then described the 
value of the separate subject catalog at Amherst College 
with its special reference to printed subject bibliogra- 
phies.^ For 30 years his question remained unanswered. The 
discussion was reopened in 1935 when Donald Coney raised a 
similar warning when he asked "Is the alphabetical catalog 
best arranged in one alphabetical sequence?" The question 
was raised not "for its academic value, but by a quest for 
the solution to the problem of simplifying, for lay use, the 
standard dictionary form of catalog." He was convinced "that 
the catalog confuses the user with a wealth of detail in 
unfamiliar form."^° This opinion was supported in a study by 
Margaret C. Brown in which graduate students using the sub- 
ject catalog were observed. One particular student was seek- 
ing information on the subject "rural recreation." 

Next the student went in search of any subject 
which began with the word "rural." Here sev^al 
titles relative to the subject were found. The 
student was highly pleased with this development^^ 
but quite unaware that these were title entries. 

The divided catalog has been turned to as an answer to 

making the library catalog more usable. Certainly extensive 

research is needed to determine what the patron looks for in 

^Fletcher, 0 £. cit . 

^^onald Coney, "The Librarian and the Catalog," 
Bulletin , 29 (September 1935), 593-94. 

^^Margaret C. Brown, "The Graduate Student's Use of the 
Subject Catalog," College and Research Librarie_S/ 8 (July 
1947), 203-08. 
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a catalog. This research may produce necessary changes in 
the rules of descriptive cataloging, in the principles of 
subject headings, or simplification of present filing codes. 
These shortcomings, however, are not automatically corrected 
by dividing the catalog. 

A summary of the literature on the divided catalog has 

12 13 

been compiled, at various times, by Pettee, Grosser, 

14 15 ^ 

Tauber, and more recently by Hines and Harris. In re- 
viewing the literature it becomes evident that the published 
reports, studies and articles on the divided catalog fall 
into one of three general categories: (1) opinions of li- 

brarians and patrons who have had experience with a divided 
catalog; (2) studies on the use of the catalog; and 
(3) general evaluations of the divided catalog with uncom- 
mitted points of view. Of the studies into the use of the 
catalog, none attempted to establish a causal relationship 
between the type of arrangement and the successful location 
of subject materials by the patron. Tauber, in fact, was 
led to remark that the evidence from such research was 

Julia Pettee, Subject Headings (New York: H. W. 

Wilson Co., 1947), pp. 185-86. 

^^Dorothy Grosser, "The Divided Catalog: A Summary of 

the Literature," Library Resources and Technical Services , 2 
(Autumn 1958), 238-52. 

^^Tauber, 0 £. cit . , pp. 92-101. 

^^Theodore C. Hines and Jessica L. Harris, Computer 
Filing of Index, Bibliographic and Catalog Entries 
(Newark, N.J. : Bro-Dart Foundation, 1966), pp. 105-06 . 
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inconclusiv© and that "th© prsssnc© of gu©ss— work and 
rationalization ... suggests that further study of the 
arrangement of the catalog is desirable." 

The problem, therefore, is to determine if the division 
of the catalog actually does result in a difference in the 
user's ability to consult the catalog. The hypothesis to be 

tested may be stated as follows: 

Assuming all other factors are equal, sub- 
ject searches through a file containing subject 
entries alone will produce more pertinent refer- 
ences and less inappropriate references than 
identical searches using a file combining all 
entries into a single (dictionary) sequence. 

That is to say, it is believed that catalog users do not com- 
pletely understand present devices for indicating conceptual 
differences (i.e., the same term or terms used to indicate 
both subjects and non-subjects) such as capitalization of 
subject entries or the use of red type. Hence, those 
changes introduced to make such differences more explicit 
should increase the effective use of the library catalog. If 
this is true, then the subject entries in a dictionary ar- 
rangement that are not in conflict with similar or identical 
non-subject entries should be retrieved as successfully as 
if they were contained in a separate file of subject entries. 

Measuring Effective Use 

Measuring the effective use, let alone improved use, of 
the library catalog presents some problems. Lilley, in an 

^^Tauber, o£. cit . , p. 98. 
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excellent discussion of the measurement of catalog use, re- 
marked that such measurement could only be made in terms of 
the function of the catalog. 

... as soon as the function of the catalog 
has been determined, and generally agreed to by 
the profession, we can use the statement of 
function as a standard of judgment that will let 
us derive estimations of value from measurements 
of use. ... at the moment we have no clear-cut 
understanding of the catalog's true function. 

There exists a fairly extensive body of literature which 
is centered around the discussion of the problem of construct- 
ing a card catalog solely as a "finding list" as opposed to 
the catalog as a comprehensive "bibliographical tool." As 
long as this argument is centered around this either/or 
dichotomy, it is difficult to arrive at any solution. The 
problem is not a matter of kind but, rather, of degree. 

Consider the patron who poses the question "Does the 
library contain such-and-such a document?" In order to search 
the catalog, the request must be made in terms of some con- 
venient descriptive tag. This tag may be an author, or 
title, or some other information which the user assumes to be 
in the catalog. The cataloger, on the other hand, is faced 
with the problem of having a document in hand and wishing to 
indicate that it is now a part of the local collection. To 
do so, he must describe it and does so by selecting one or 
more descriptive tags. These too may be the author's name, 

Oliver Linton Lilley, "The Problems of Measuring 
Catalog Use," Journal of Cataloging and Classification , 10 
(July 1954), 127. 
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or the title of the document, or some other information which 
is deemed important enough to distinguish one document from 
another • 

The number and completeness of the tags selected by the 
cataloger are based upon two factors. First, the cataloger 
realizes that the document is ordinarily physically accessible 
and can limit his description to a greater degree than can the 
bibliographer who must assemble together items that may not 
be accessible at one single place or may not actually exist 
(i.e., an "ideal copy"). In reality this procedure does not 
assign any function to the catalog that is not traditionally 
expected. Writing about what is probably the largest card 
catalog in this country, Dubester said of the Library of 
Congress catalog: 

The present catalog in its dictionary form 
provides for the exercise of certain very spe- 
cific functions. It serves primarily to deter- 
mine whether a known and identifiable work is 
in the collection. Further, and as a very spe- 
cific result of the ALA rules for author and 
title entry and our pattern of subject analysis, 
we are able to answer the following: give me 

all the works of such-and-such an author; give 
me all editions of forms of such-and-such a 
work; and what do you have on such-and-such a 
subject and on related subjects.!® 

This is, essentially, a synthesis of the separate functions of 

the author-title catalog as expressed at the 1961 "Conference 



^®Henry J. Dubester, "Studies Related to Catalog 
Problems," Library Quarterly , 34 (January 1964), 100. 
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on Cataloguing Principles''^^ and the function of the subject 

20 

catalog as outlined by Coates. 

The second factor, which the cataloger must recognize, 
is the amount of information the patron is likely to possess. 
This problem quite naturally leads to the suggestion that, 
in order to make such a judgment, one must know something 
about who uses the catalog and for what purpose. In re- 
sponse to this question, numerous catalog use studies have 
been undertaken. 

One of the earliest advocates of a systematic, experi- 
mental approach to the study of catalog use, William M. 

Randall, prepared the theoretical framework within which most 
of the subsequent studies were undertaken: 

The problem, then, is to fit our catalogs to 
the patrons we serve. This cannot be done by any 
study of the catalogs themselves, or of the rules 
by which they are made; neither can it be done 
merely by an examination of the books to be cata- 
loged. It can be done only by an intelligent 
study of the patrons themselves; their mental 
equipment, their background, and their needs. ^1 

There are over 40 catalog use studies which have been re- 
ported in the literature of librarianship and which represent 



^^International Federation of Library Associations. 
International Conference on Cataloguing Principles, Paris, 
9th-18th, October, 1961. Report (London, Organizing Com- 
mittee of the International Conference on Cataloguing 
Principles, 1963), pp. 91-92. 

J. Coates, Subject Catalogues; Headings and 
Structure (London, Library Association, 1960) , p. 19. 

^^William M. Randall, "The Uses of Library Catalogs: 
A Research Project," Catalogers' and Classifiers' Yearbook 
Number Two (1930) (Chicago: American Library Association, 

1931), pp. 31-32. 
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a wide variety of catalogs and patrons subjected to study 

and evaluation. Most of this literature has been ably sum- 

22 23 24 

marized by such writers as Stevens, Tauber, and Frarey. 
These post-1930 studies have indicated a wide variation in 
users, their ability, their concepts of the proper role of 
the catalog in the complete bibliographic system, and dif- 
ferences in the patrons' background (such as formal instruc- 
tion in the use of libraries and catalogs, the amount of 
experience, the influence of previous "successful uses, 
etc.). Many of the studies have also included a discussion 
on the effectiveness of the catalog. The method of measur- 
ing effectiveness has essentially been that of determining 
how frequently the patron is satisfied in his search for 
some material. According to this measure, Frarey is able to 

report a median of 70 percent based on studies which report 

25 ^ 

successes that range from 50 to 95 percent. In a more 
recent study, Morris reports that the effectiveness of one 
college library catalog — in terms of subject searches — was 

^ ^Holland E. Stevens, A Summary of the Literature on 
the Use Made by the Research Worker of the Unive rsity Library, 
Catalog. (University of Illinois Library School, " Occas lonal 
Papers" no. 13, Urbana, 1950) . 

^^Tauber, 0£. cit . , pp. 65-101. 

^"^Carlyle J. Frarey, "Studies of Use of the Subject 
Catalog: Summary and Evaluation," in Maurice F. Tauber, ed. , 

The Subject Analysis of Library Materials (New York: School 

of Library Service, Columbia University, 1953), pp. 147-66. 

^^Carlyle J. Frarey, "Subject Headings," ^ Shaw, 
op . cit ., Vol. 1, Part 2, pp. 49-50. 
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96 percent. As a result of these studies and their findings 

many of the investigators have recommended that there is a 

great need for "more and better instruction in catalog use 

27 

among all classes of patrons." 

The basic difficulty with attempting to apply this 

recommendation is that there is no conclusive evidence that 

would help to establish the appropriate level of knowledge 

and familiarity that such instruction must attempt to reach. 

Not even the professional library education program would 

seem to guarantee adequate preparation if there is general 

applicability to Jackson's observation that "not all staff 

members were more skilled than the patron at using the 

..28 

catalog. " 

While the general findings of the numerous catalog use 
studies have been subjected to criticism and found wanting, 
there has seldom been an effort to determine why these 
studies should produce such inconclusive data when discussing 
the effective use of the catalog. The very assumption that 
the methods employed to conduct these studies have been 
appropriate and adequately designed is subject to question. 
Two of the more perceptive critics of catalog evaluation 

^^Mary Eugenia Morris, "A Study of the Use of the Card 
Catalog at Western Carolina College Library." (Unpublished 
Master's thesis. School of Library Science, University of 
North Carolina, 1966), p. 39. 

^^Frarey, "Studies," p. 163. 

^®Sidney L. Jackson, Catalog Use Study; Director '_s 
Report , ed. by Vaclav Mostecky (Chicago: American Library 

Association, 1958), p. 2. 
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studios ——Mortiiuer Taubo and Olivor L. Lilley— bavO/ indood, 
cast doubt about the methods used in conducting such studies 
and have proposed alternate approaches- Lilley's proposal 
is principally one of comparing existing subject catalogs 

against an ideal catalog: 

If we had an ideal subject catalog, or what 
would*~He just as good, an exact and complete de- 
scription of ,an ideal subject catalog, then we 
could compare the actual instrument with the 
model, detail by detail, and develop a precise 
statement of their differences- - - - 

The crux of our problem then is to find a 
way to give concrete expression to something the 
library profession as a whole might accept as 
the "ideal" toward which the subject catalog 
should be expected to approach. 2 9 

Lilley then continued by outlining a five-point program that 

would attempt to achieve this ideal- He conceded that his 

plan was one step further removed from direct evaluation of 

the catalog but he felt it would be "achieving the same goal 

of improvement through a more basic study of appropriate and 

30 

desirable functions, and of fundamental philosophy-" 

Taube, in his criticism of general attempts to evaluate 
"information systems," has pointed out the interrelationship 
between the design and characteristics of the system itself 
and the satisfaction of the user, naming these internal and 
external criteria Although realizing the importance of 



^ ^Oliver L- Lilley, "Evaluation of the Subject Catalog; 
Criticisms and a Proposal," American Documentation , 5 (April 
1954), 47c 



^°Ibid., 49. 
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using both criteria for evaluation, Taube concluded that at 
that time such evaluation could not be accomplished. Instead, 
he proposed that fifteen characteristics, which he identified 
as implicit evaluative criteria in previous studies, could 

be enumerated and that these fifteen might be agreed upon 

. . 32 

"as criteria or measures of consumer satisfaction." Thus, 

Taube felt, an evaluation of the internal criteria was a 

valid procedure since it accounted, by agreement, for the 

criterion of user satisfaction. 

The proposals of Lilley and Taube are not really very 
different; both attempt to establish a standard of some type 
against which comparison might be made. The basic difference 
seems to be in the method of achieving this standard rather 
than in the method of evaluation. Both proposals, however, 
suffer a critical shortcoming as a method for evaluating 
catalogs — or any other type of index. The two are dependent, 
as noted, upon an ideal or basic standard. (Absence of such 
agreement, of course, makes the approach impossible.) Assum- 
ing that such agreement can be reached, the index system 
must then be tested to determine if the underlying assumption — 
that internal criteria (or professional judgments) are, indeed, 
an accurate measure of consumer satisfaction — is valid, and 
this requires a user-oriented approach. Neither proposal 
seems to suggest any marked improvement over the methods of 
previous catalog use studies. 

^^Ibid. , p. 101. 
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The Experimental Approach to Measuring Effective Use 
A criticism of previous studies must begin with an 
examination of the principle which served as a directive for 
the studies. A review of the most significant studies in- 
dicates that there is either a direct or indirect connection 
with Randall's thesis as presented in his 1930 address. As 
a method of finding out who uses the catalog and for what 
purpose, his proposal is quite feasible. When it is made to 
serve as the focus for determining how well individuals use 
the catalog, what problems they encounter, and what improve- 
ments will erase these problems, the traditional catalog use 
studies are completely inappropriate. Whether intentionally 
or not, Randall's thesis has been paraphrased in these 
studies to read: 

Give us an idea of the nature of the 
typical user and we can construct a catalog 
which anyone should be able to use. 

The most obvious problem in this method of evaluating 
the effectiveness of the catalog is that of developing an . 
acceptable definition of a typical user. Such a user may be, 
at least by definition, one who has little trouble in using 
the catalog, or one who has little success, or some arbitrary 
standard between these two levels of performance. Unfortu- 
nately, if one must also consider the possible ranges in age, 
education, occupation, and other factors, the meaningfulness 
of the term "typical" ceases to exist. That this is obvious 
today is primarily due to the results of the studies being 
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criticized, and is sufficient justification for having done 
them. At the same time, they illustrate the futility of 
continuing this approach as a method for evaluating the 
catalog . 

A second, and more critical, shortcoming of the previ- 
ous studies has been their failure to recognize the very 
interaction that Taube mentions. Most studies have at- 
tempted to generalize a situation that includes a varying 
patron (and his inevitable individual differences) , a vary- 
ing catalog and a wide range of questions posed by the user. 
The measure of effectiveness, therefore, is actually that 
of a patron-catalog-question system and should not be 
attributed to any single element — the catalog — unless 
certain factors are controlled. For example, the searches 
utilized to test catalog users were generated by the patron 
rather than the investigator in most instances. Since there 
is no existing evidence that these represented all types of 
problems that may be encountered, the calculated level of 
effective use is applicable only to specific sections 
consulted and may not be generalizable to the catalog as a 
whole . 

Furthermore, there is the matter of the catalog itself. 
The assumption that the catalog variable has been held con- 
stant in each of the studies is open to question. The com- 
plexity of any given section of the catalog will vary in 
relation to the particular part (and size) of the collection 
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represented, the complexity of the filing rules applicable 
to that section, and the possibility that the term selected 
to be searched may appear in a number of variant forms (i.e., 
main, subject added, or non-subject added entries) . If 
various parts of the catalog are more difficult to use than 
others (e.g., the "U.S." drawers), then the differences in 
successful use may only reflect the accidental occurrence 
of selecting patrons using different parts of the catalog. 
Thus, depending on the search, patrons may have been using a 
fairly simple bibliographic tool or an exceptionally complex 
one, even though it was one and the same catalog. 

Hence, a more fruitful approach would be to restate the 
user-oriented method of study in rather different terms. The 
general thesis then becomes: 

If we have a known level of effective use 
for a given patron and a given type of library 
catalog, what changes in bibliographic entry, 
formatting, increases in cross-references, etc., 
will yield a significant increase in the patron's 
effectiveness level? 

Such an approach combines the advantage of testing directly 
the user's satisfaction (and ability), provides evidence 
of the effect of the interplay between internal criteria 
(the catalog) and external criteria (both the user and the 
problem-question) and is a comparison between a basic standard 
(the known level of effective use) and the level of effective- 
ness using a modified catalog. Rather than establishing an 
"ideal" catalog, however, it is possible to take any existing 
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catialog, determine how well the patron is able to use it, and 
then have the patron replicate the searches using the catalog 
after it has been changed. 

It would seem that this method is applicable to almost 
any type of comparison, given the necessary controls over 
all variables in the system. At the present time no single 
study can adequately cope with all the possible combinations 
that reflect alternate forms of the various internal and 
external criteria. There is a need for a complete series of 
studies in which each element is introduced as an experimental 
variable and the influence on effective use measured. It is 
within this framework that the design for testing the hypothe- 
sis concerning the improvement of catalog use by division was 
developed. 

Summary 

The merits of the divided catalog versus the dictionary 
catalog have been discussed in the literature of librarian- 
ship for over 30 years. A review of this literature revealed 
that the evidence in support of either type of arrangement is 
inconclusive. The purpose of this study was to investigate 
the comparative effectiveness of a divided catalog and a 
dictionary catalog in facilitating subject searches. The 
problem of measuring effective use, and certain methodological 
shortcomings of previous research, led to the development of 
the comparative method of study. An experimental approach 
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was outlined in which a patron (a constant) would make the 
same search (also a constant) using, first, a catalog in 
dictionary arrangement and then the same catalog divided 
into author-title and subject sequences. Chapter II is a 
discussion of the problems, limitations, and design of a 
practical study developed within this theoretical framework. 




CHAPTER II 



DESIGN AND PROCEDURE 

The purpose of this study was to determine if dividing 
a traditional dictionary catalog to create a separate file for 
subject entries would result in an increase in the effective 
use of library catalogs. A comparative method for making such 
an evaluation was developed in Chapter I. The purpose of this 
chapter is to describe the influence of practical limitations 
on the final design of the study. Control of the variables 
that must be considered in an experimental approach to test- 
ing the hypothesis are discussed. In addition, the effects 
of various underlying assumptions and the desired analysis 
to be made are considered in relation to the design and 

procedure . 

Design of the Experimental Approach 
The causal element .— Difficulty in the use of the cata- 
log may be due to a number of reasons. Occasionally patrons 
expect to -locate material that is not traditionally indexed 
by the library catalog. Incorrect bibliographical informa- 
tion may also be the cause for failure in locating appropriate 
material. Frequently, failure can be attributed to the 
patron's inability to formulate the correct search strategy 
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by not selecting the appropriate term or synonym. In other 

cases the term selected is not of the appropriate level of 

specificity to isolate the necessary references. This problem 

of selecting the appropriate search tern has been emphasized 

1 2 

in such studies as those conducted by Malcolm and Lilley. 
Dividing the catalog into two or more separate files will 
not, of course, alleviate all these difficulties. Division, 
in fact, would seem to result only in making the subject 
approach more explicit and in reducing the confusion between 
subject and non-subject entries. That confusion is a source 
of failure can be indicated by considering the types of cards 

to be found in a card catalog. 

The catalog is composed of three basic types of cards: 

3 

main entries, added entries, and references. Subject head- 
ings are primarily added entries and reference ("see" and 
"see also") entries. By the nature of the Various rules of 
cataloging, similarities between the subject of a book and 
the title of that book are suppressed in favor of subject 

^Roberta Suits Malcolm, "The Student's Approach to the 
Card Catalog: A Study Based on a Survey of Student Use at the 

Library of the University of Pittsburgh ..." (Unpublished 
Master's thesis, Carnegie Library School, Carnegie Institute 
of Technology, 1950) , p. 9. 

^Oliver L. Lilley, "Evaluation of the Subject Catalog; 
Criticisms and a Proposal," American Documentation , 5 (April 
1954), 41-42, 51-60. 

^The definitions of the terms are those est^lished at 
the International Conference on Cataloguing Principles. See 
International Federation of Library Associations. International 
Conference on Cataloguing Principles, Paris, 9th-18th October, 
1961. Report . (London, 1963), p. 115. 
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added entries. The possible occurrence of misunderstanding 
between a subject added entry and a title added entry is, 
therefore, reduced. Other subject headings, however, are of 
a form that is identical to main or non-subject added entries. 
Hence, reference to material by or about an individual, 
society, institution or governmental agency will be shown by 
using the same terms for the heading whether they are main 
or secondary entries. To emphasize the difference between 
these identical headings being used to signify different con- 
cepts, two general devices are used. One device is to vary 
the typographical presentation, resulting, in most cases, in 
the indication of subject added entries in capitals or red 
type. (A variation of this approach is the one found in 
printed indexes where the subject is indicated by some 
heading— frequently printed in italics — while other types of 
added entries are suppressed in favor of a cross reference 
to the main entry.) The second device is to treat each 
heading and its corresponding conceptual representation as 
a separate file. In the dictionary catalog this results in 
the filing of subjects after the main and added entries, in 
the divided catalog it results in the establishment of a 
physically separate file. 

Some evidence and much testimony would indicate that 
these devices are often too subtle for the lay user of the 
catalog to recognize unless the device is made more explicit 
as in the case of the divided catalog. Considering the 
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number of possible conflicts, this becomes a serious problem 
in large or highly specialized catalogs • The fact that the 
divided catalog does, at least in theory, tend to make the 
subject references more explicit is the reason (i*e., the 
causal element) for predicting that the divided catalog 
would be more effective to use than a dictionary catalog. 

If the same person makes identical subject searches in two 
catalogs— one divided and one in dictionary arrangement— —it 
can be assumed that the difficulties the patron would have 
\70uld be common to both catalogs except for one area* in the 
divided catalog, title and other conflicting entries could 
not be confused with appropriate subject entries. 

The search-problem as a variable . — The traditional 
method for securing data on catalog use has been to observe 
patrons at the catalog and to note what the person was 
looking for. Some expression or judgment that the patron 
found what was needed was deemed the criterion for success 
or failure, indicating that the catalog was or was not an 
effective bibliographical tool. This measure of effective- 
ness assumes that a given patron who may successfully use 
the catalog in searching for one item will not fail com- 
pletely on the next search even though subsequent searches 
roquire coping with different degrees of complexity. 
Under such circumstances it is questionable to attribute 
success or failure to the catalog alone rather than to a 
variety of factors. 
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To give meaning to any statement that one catalog is 
"better" or "easier" or more "effective" to use, it is neces- 
sary that the comparison be made across a broad spectrum of 
representative problems or at least about identical types of 
searches. Having patrons search the catalog for a request 
presented by the investigator would achieve the desired 
control. In order to test the basic assumption that diffi- 
culties actually do vary according to the type of question — 
and the complexity of the catalog encountered— the development 



of a series of problems is necessary. 

First, the selected search-problems should include a 
number of entries that represent different concepts (that is, 
identical headings representing main, non-subject and subject 
added entries) as a means for testing the original hypothesis. 
Furthermore, to determine if this confusion of concepts is the 
actual source of difficulty, and not representative of diffi- 
culty in dealing with subject searches in general, additional 
search— problems should be included that represent searches 
under the more traditional form of subject heading. The 
variety of forms, as identified by Haykin and more precisely 
categorized by Eaton, ^ would be more than could be included 
within a reasonable problem-solving exercise and will have to 
be limited. 

^David Judson Haykin, Subject Headings; A Practical 
Guide (Washington, D.C.; U.S. Government Printing Office, 
1951) , pp. 21-25. 

^Thelma Eaton, Cataloging and Classification; ^ 
Introductory Manual (3rd ed.. Champaign, 111.. Distributed 
by the Illini Union Bookstore, 1963), p. 131^ 
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Finally, dividing the catalog to increase effective- 
ness in making subject searches should not result in a cor- 
responding decrease in the ability of patrons to locate known- 
items (for which an author, title, editor, etc., is known). 

To test this assumption that division does not decrease the 
effective use of the catalog to search for a known-item, a 
few problems for which the author and title are given should 
be included. 

The catalog variable . — Ideally, an evaluation of any 
type of catalog with any type of user-universe can be under- 
taken. The procedure, in simplest terms, would be to have 
patrons conduct a series of controlled searches using a 
specific catalog, to make whatever change in the catalog is 
deemed important for producing an improved catalog, and then 
to have the patrons replicate the searches. A comparison of 
the successes between th<^! two catalogs would isolate the in- 
fluence of catalog manipulation on the effective use of the 
catalog. This procedure, however, has certain weaknesses 
because there is no control over the maturity of the user 
(the measures are on individuals whose previous exposure to 
the two catalogs varies) and it creates the difficult problem 
of developing two sets of questions identical enough to 
measure the same searches but sufficiently different to 

reduce the effects of memory. 

A more practical limitation is that such a study would 
result in certain hardships in the actual operation of any 
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given library — both in terms of cost to the institution and 
in the inconvenience to the general patron. If it is im- 
practical to utilize an existing catalog to make temporary 
changes in arrangement to use the "before-after" technique 
of comparison, it should be possible to find two catalogs 
that are relatively similar in all respects except arrange- 
ment. Furthermore, since the type and number of searches to 
be made are to be controlled by the investigator, it is 
possible to limit, or at least to estimate, what sections of 
the catalog will be used, making it necessary to match only 
sections rather than the whole of the catalogs. 

The patron as a variable . — The introduction of two cata- 
logs at separate institutions, on the other hand, poses a 
problem in maintaining control over the participant — since the 
hypothesis is stated in terms of one person's success (or 
failure) in using the catalog. One approach to this problem 
is to attempt to match individuals at the two institutions and 
to treat the results as those of one person, the difference 
being attributed to the experimental variable — the difference 
in arrangement of similar catalogs. Matching individuals 
presents obvious difficulties, but as Selltiz points out; 

The more precise the matching and the greater 
the number of factors on which matching is to take 
place, the greater the number of cases for which 
no match is available. Fortunately, however, rele- 
vant factors are often so interrelated that match- 
ing on one factor brings with it partial matching 
on other factors; there is a "diminishing return" 
as additional factors are controlled. « 

^Claire Selltiz ^ , Research Methods in Social 

Relations (Rev. one-vol. ed. ; New York: Holt, Rinehart and 

Winston, ^1959) , p. 105. 
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The limitations of previous catalog use studies are no 
more evident than when faced with the problem of predicting 
what factors are interrelated and what would be the appro- 
priate one or two variables to use to match individuals. One 
important factor is that the two user-universes must be as 
similar as possible. In addition, it is assumed that facility 
in using the catalog is related to experience and familiarity 
in using any single catalog. For the selected user -universe 
of undergraduate students, it was determined that this 
factor of his familiarity-experience could be measured by 
obtaining information about the student's class standing and 
the frequency of use of the main catalog. The validity of 
this assumption could be tested by analysis of the resulting 
test scores against other personal and educational charac- 
teristics after the collection of the data. 

Scoring the results .— The scoring of the searches was 
determined by the objective of the study. Previously, tests 
of effective use have been based on some degree of satisfac- 
tion on the part of the user that what he has located is rele- 
vant to his need. This concept of relevancy poses more 
difficulties. While it is fairly simple to determine whether 
or not someone has located a known-item, it is more diffi- 
cult to make a judgment as to the appropriateness of a refer- 
ence when the subject approach has been used. Furthermore, 
what may be a relevant document for one individual may be 
irrelevant for a second, even though they both may have 
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selected the same index term to make the search. Therefore, 
it was intended that this study would deal with the patron s 
ability to search the catalog for appropriate cards rather 

than for specific documents. 

The judgment as to relevancy is fairly objective since 
the reference is defined as relevant only if it contains the 
exact subject heading requested. The patron will then indi- 
cate all, none, or some, pertinent references, and none or 
some non-pertinent references. In order to make the results 
comparable it was proposed to compute a "success ratio for 
each search and to compute a mean success score for each in- 
dividual. The success ratio (S) was computed by using the 
following formula: 

let r = the number of relevant references 
retrieved by the participant; 

k = the total number of known relevant 
references in the file; 

t = the total number of references re- 
trieved by the participant; 

then: S = (r/k) x (r/t) = rV^t. 

In addition, when the number of relevant references retrieved (r) 
is zero, the success ratio, by definition, was scored as zero. 

A hypothetical example may be used to demonstrate the scores 
that can result from various procedures. 

Consider a file of cards which contains ten main entries, 
ten non-subject added entries and ten subject added entries 
for a heading such as "U.S. Library of Congress." The number 
of relevant references about (rather than by) the Library of 
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Congress would be ten (k = 10) . The patron who selects all 
ten cards would receive a score ofs 

S = rVkt = (lO)VuO) (.10) = 1.000. 

If the participant mistakes the ten added entries for 
subject entries and selects the added entries as well as the 
ten subject added entries, his score would be: 

S = rVkt = {lO)V(lO) (10+10) = 0.500. 

If for some reason, the patron selects only five of the 
relevant cards (and no other) his score is also: 

S = rVkt = (5)^{10){5) = 0.500. 

It must be emphasized that this success score has mean- 
ing only in relation to determining the relative success of the 
individual in differentiating between the desired relevant 
references and the non-pertinent (i.e., conflicting) references 
that may confuse him. Thus, the two scores above (for which 
S = 0.500) represent similar levels of effective use. The in- 
dividual who must search twice as many cards as there are 
relevant cards in order to locate the relevant deck of refer- 
ences must be considered no more effective in using the cata- 
log than the patron who selects only half of the relevant 
deck. The reason for achieving a score of less than 1.00 in 
one case is presumably different than the reason for receiving 
a score of 0.500 in the other. This relationship of score to 
cause of partial, rather than complete, success is examined 
in detail in Chapter V. 
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The expected differences between the matched pairs can 
be tested for statistical significance using the t-test for 
difference of means. ^ All statistical tests for hypothesis 
testing will be made at the .05 level; that is the probability 
of rejecting a true hypothesis will be set at 5 percent for 
all statistical tests. 

The Wisconsin and Illinois Catalogs 

Because of certain similarities — and geographical 
proximity — two large universities were selected as the focus 
for this study — the University of Illinois (with a dictionary 
catalog) and the University of Wisconsin (with a divided cata- 
log) . Both schools are large, midwest, state-supported uni- 
versities of national reputation o Table 1 shows how similar, 
although not identical, the two schools are in respect to the 
distribution of the study body according to class standing. 

On the other hand, there is a rather large discrepancy between 
the size of the collections at Illinois and Wisconsin which 
could have an effect on the attempt to locate comparable 
sections in the two catalogs. Illinois' collection was 
listed as 4,083, 634 volumes for 1965-66 as compared to 

Q 

Wisconsin's 1,746,321 volumes. The Wisconsin holdings do 
not include the collection housed at the Milwaukee campus nor 

^Allen L. Edwards, Statistical Methods for the Behavioral 
Sciences (New York; Holt, Rinehart and Winston, 1954) , 
pp. 278-82. 

^American Library Association. Library Administration 
Division, Library Statistics of Colleges and Universities , 
1965-66; Institutional Data (Chicago; American Library 
Association, 1967) , pp. 26, 82 . 
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TABLE 1 

DISTRIBUTION OF STUDENTS BY CLASS AT THE 



UNIVERSITY OF ILLINOIS 
WISCONSIN, FALL 


AND THE 
SEMESTER 


UNIVERSITY OF 
1966-67 






Illinois 


Wisconsin 


Class 


Number 


Percent Number 


Percent 


Undergraduates 


Freshmen 


6,324 


21.7 


5,847 


18.8 


Sophomores 


5,144 


17.7 


6,484 


20.8 


Juniors 


4,542 


15.6 


4,849 


15.6 


Seniors 


4,583 


15.7 


4,311 


13.9 


Others* 


193 


0.7 


394 


1.3 


Subtotal 


20,786 


71.4 


21,885 


70.4 


Professional 


836 


2.9 


1,013 


3.2 


Graduates 


7,498 


25.7 


8,222 


26.4 


Total 


29,120 


100.0 


31,120 


100.0 



*Category includes "unclassified" and "irregular" stu- 
dents at Illinois; "special" students at Wisconsin. 



SOURCE: Illinois data provided by the Office of Admissions 

and Records (11/8/66) ; Wisconsin data provided by 
the Office of the Registrar (1/20/67) . 

do they reflect the collection of the State Historical Society 
of Wisconsin which may be considered an adjunct to the 
Wisconsin collection. Discounting some 332,500 volumes at 
the Chicago campuses, the Illinois catalog indexes more than 

twice as many volumes as Wisconsin's. 

A preliminary visit to the Madison campus, however, in- 
dicated that many of the search-problems selected at Illinois 
(as described below) were comparable in size of file to those 
at Wisconsin. The largest discrepancies occurred among those 
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entries that were most likely to represent local interests 
(e.g., the file size for references to documents by and 
about Stephen A. Douglas was expectedly larger at Illinois 
than at Wisconsin) . 

In regard to the physical size, the University of 
Illinois' alphabetically arranged dictionary catalog occupies 
4,800 drawers and the University of Wisconsin's catalog con- 
sists of 1,232 subject drawers and 3,439 drawers in the 
author-title section. The Wisconsin catalog, according to a 
recent audit, consists of over 3,300,000 cards (approximately 

924,000 subject cards and 2,407,000 author-title cards) as 

^ , 9 

compared to well over 5,000,000 cards in the Illinois catalog. 

Far more important are the internal characteristics of 
the two catalogs. As might be expected, both libraries 
follow standard rules of cataloging, frequently using printed 
Library of Congress cards to supplement their own original 
cataloging. No difficulty was expected because of minor dif- 
ferences in the two catalogs: 1) subject added entries are 

indicated by black type on unit cards at Wisconsin while 
Illinois displays subject added entries by red type; and 
2) added entries are interfiled in the Wisconsin author-title 
catalog (as per ALA rule 25a) while at Illinois added 



^Data for Wisconsin catalog based on a recent audit and 
provided by Mr. LeRoy D. Ortopan, Chief of Catalcpging, on 
February 9, 1967. No recent audit of the Illinois catalog has 
been undertaken; the conservative estimate of five million 
cards was verified by Miss Betty Croft, Catalog Librarian. 

Rules for Filincr Catalog Cards (Chicago: 

Amer ican Library Association, 1942) , p. 25. 
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entries are filed after main entries (and before subject 
added entries) resulting, generally, in three separate files 
for each specific heading. 

One important difference in filing, however, was con- 
sidered significant enough to be a potential cause of some 
difficulty in making comparisons. In particular, the rule 
affects the filing order of subordinate agencies and units 
for headings under geographical names. The "United States" 
sections of the two catalogs is one example which had direct 
applicability to the study. At Wisconsin, subordinate agen- 
cies (e.g., U.S. Civil Service Commission) are filed after 
the general heading "United States" and its appropriate sub- 
divisions. At Illinois, these subordinate agencies are 
treated, for filing purposes, to be indistinguishable from 
subject subdivisions. For example, subject entries for 
various headings under "United States" appear in the two 
catalogs in the following order; 



Thus, the search for subject cards for the entry "U.S. Civil 



Wiscons in 



Illinois 



U.S.— ALTITUDES 
U.S. —BIBLIOGRAPHY 
U.S.— CIVILIZATION 
U.S. —FOREIGN RELATIONS 
U.S. — HISTORY 

U.S. — POLITICS & GOVERNMENT 
U.S. — STATISTICS, VITAL 
U.S.— TERRITORIAL EXPANSION 
TT cj ARMY 

U ! S ! ARMY — BIBLIOGRAPHY 
U.S. LIBRARY OF CONGRESS 
U.S. WEATHER BUREAU 



U.S. — ALTITUDES 
U.S. ARMY 

U.S . ARMY— BIBLIOGRAPHY 
U . S . — B IBLIOGRAPHY 
U . S . — C IVI LI Z AT ION 
U.S.— FOREIGN RELATIONS 
U.S.— HISTORY 
U.S. LIBRARY OF CONGRESS 
U.S. — POLITICS & GOVERNMENT 
U.S. —STATISTICS, VITAL 
U.S.— TERRITORIAL EXPANSION 
U.S. WEATHER BUREAU 



Service Commission" should be made between "U.S. Army — 



Bibliography" and "U.S. Library of Congress" in the Wisconsin 
catalog. In the Illinois catalog, "U.S. Civil Service Com- 
mission" would be found between the entries for "U.S. — 
Bibliography" and "U.S. — Civilization." 

Rather than eliminating such examples from the search- 
problems, it was decided to include them and to determine if 
this was the cause of success or failure more frequently at 
one school than at the other. If the data indicated that 
this difference did, indeed, affect the results, the problems 
would be deleted from the student's mean success score and 
would be analyzed separately. 

Development of the Search-Problem Exercise 

The actual selection of the search -problems was achieved 
by random sampling of the University of Illinois library 
catalog. The objective of the sampling was to obtain a list 
of personal, corporate and uniform entries that could be com- 
pared with the Wisconsin catalog. Without any knowledge of 
how many such headings would be found or how many would be 
necessary for matching with the divided catalog at Wisconsin, 
it was decided to obtain 480 random numbers (10 percent of 
the total number of drawers) . It was also determined that 
the majority of drawers would not have more than ten inches 
of cards and a second set of random numbers was obtained to 
represent one inch of cards per drav/er. Five hundred and 
twenty-five four-digit numbers and the same amount of single- 
digit numbers were drawn and combined — the first 480 unique 
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Goitibinations wsr© us©d in th© sanipling* Thus, th© nuit\b©rs 
•'9" and "0029" indicat©d that th© ninth inch of draw©r 
nunib©r 29 was to b© audit©d« Th© initial audit produc©d 283 
p©irsonai, corpoirat© and uniforiti ©ntiri©s* In ordar to jnininiizs 
th© nuinbar of ©ntrias that might Eiot b© found in the smaller 
divided catalog, it was decided to reduce this list. The 
list was reordered into seven categories: 

a) Main Entries only; 

b) Non-subject Added Entries only; 

c) Subject Added Entries only; 

d) Main and Non-subject Added Entries; 

e) Main and Subject Added Entries; 

f) Non-subject Added and Subject Added Entries; 

g) Main, Non-subject Added and Subject Added Entries. 

Entries were then selected (or excluded) from the list 
on the basis of the following criteria: 1) entries represented 

by one or two cards were not selected if they were no dif- 
ferent in form than others within that group; 2) entries 
representing obscure individuals or organizations were deemed 
to be no more representative than other entries in their 
category; and 3) selected entries were considered to be of a 
type that could more easily be stated as a search request 
than other entries in their category. The first two criteria 
could not be considered, within the nature of this study, to 
have had any appreciable effect on the final search-problems. 
The latter criterion, however, did result in the deletion of 
uniform headings which would have been awkward to state as 
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problems. Such headings as "Georgia. Laws, statutes, etc ." 
did not lend themselves readily to the type of search- 
problems desired and were, therefore, excluded. As a result 
of this procedure, 48 personal and corporate entries and two 
examples of criticisms were left as potentially usable sub- 
ject searches. 

A list of 200 conventional subject headings was com- 
piled by sampling from the Library of Congress subject head- 
11 

ings list. The catalog of the University of Illinois was 
again audited to determine if these subject headings were 
actually used and to investigate the feasibility of including 
them as problems. Judgment as to feasibility was based on 
the desire that there be some potential conflicts between 
subject headings and similar, although not identical, titles. 
Furthermore, subject headings were selected if they seemed 
to lead to convenient solutions regardless of the possible 
interpretation of the patron. From this list only nine 
subject headings were determined to be potentially usable. 

12 

Pre-Test of the Exercises 

Prior to the final comparison of the appropriate 
sections of the two catalogs used in the study, a pre-test 
was conducted. The purpose of the pre-test was: 1) to deter- 

mine the effect, if any, of the various forms for wording the 

^ ^Subject Headings Used in the Dictionary Catalogs of 
the Library of Congress (6th ed.; Washington, D.C.: Library 

of Congress, 1957) . 

12 

For forms used in the pre-test, see Appendix A. 
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question that had been considered; 2) to determine the total 
number of questions that might be asked in a one— hour test 
period; and 3) to determine if patterns of search actually 
were similar enough to predict the general sections of the 

catalog that should be compared. 

The pre-test was conducted in September 1966 with 28 
members of the introductory class in cataloging in the 
Graduate School of Library Science at the University of 
Illinois. The exercise was conducted in lieu of a first week 
laboratory assignment in cooperation with the instructor of 
the course. Class members were instructed to sign up for a 
one-hour period at which time they reported to the investi- 
gator. Each student completed a "General Information Sheet" 
and was informed of the purpose of the exercise and the pro- 
cedure. Inasmuch as this was a pre-test intended to deter- 
mine the appropriateness of the method, students were en- 
couraged to ask questions if the instructions or the 
problems were not clear or seemed ambiguous. The partici- 
pant then conducted the series of eight searches at the main 
catalog in the presence of the investigator. As each stu- 
dent went through the problems, the investigator took note 
of the procedure and final decision. By varying the com- 
bination of searches, each of the final 59 subjects was 
searched at least once. Note was also made of the time the 
searches were begun and completed, yielding a total time 
elapsed (rather than individual times for each question) » 
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As noted before, there was some concern over the word- 
ing of the problems which led to the decision to vary the 
question during the pre-test phase. Basically, there were 
five methods of asking for similar items s 

Q- : Locate the appropriate catalog cards that in- 

^ dicate the library contains documents abou jt 

the topic ; 

: Locate the appropriate catalog cards that in- 

^ dicate the library contains documents about ; 

Q ; Locate the appropriate catalog cards that in- 
^ dicate the library contains documents abou_t 

(rather than by) ; 

Q* ; Locate the appropriate subject cards in the 
^ university card catalog for the following; 

Q- : Assume you have the text of the title listed 

^ below, locate the appropriate catalog cards 

that indicate the library contains documents 
about ; 

These basic forms of the question were combined in the follow 
ing manner for four groups of students: 



TABLE 2 

FORM OF QUESTION USED FOR FOUR GROUPS OF STUDENTS 



Type of Problem 

Conventional Subject 

Person as a Subject 

Corporate Entry as a Subject 

Criticism 



Student Group 
I II III IV 




Q Q Q Q 
2 13 4 

Q2 °4 
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It was assumed that the form and the results of the 
search -problems are independent. This was tested by the null 
hypothesis, that there is no relationship between the form 
of the question and the frequency of failure, success, or 
partial success. Each student's response was scored in a 
very general way as being completely correct, partially 
correct, and completely incorrect. A 4 x 3 contingency 
table was created and the independence of the two variables 
(form and response) tested by computing a chi-square. The 
observed and calculated expected frequencies (given in 
parentheses) are shown in the accompanying table. 

TABLE 3 



OBSERVED AND EXPECTED FREQUENCIES FOR PRE-TEST DATA 



Student Group 


Failure 


Partial 

Success 


Success 


Total 

(Rows) 


I 


34 


26 


20 


80 




(30.4) 


(26.8) 


(22.9) 




II 


19 


17 


12 


48 




(18.2) 


(16.1) 


(13.7) 




III 


10 


10 


12 


32 




(12.1) 


(10.7) 


(9.1) 




IV 


22 


22 


20 


64 




(24.3) 


(21.4) 


(18.3) 




(Column 










Total) 


85 


75 


64 


224 



The computed value of chi-square was 2.83. At the pre- 
determined level of a probability of 5 percent occurrence of 
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a Type I error and for 6 degrees of freedom, the computed 
value is not significant. Therefore, there is no reason to 
reject the null hypothesis that form (Q) and Success-Failure 
are independent and not related. The final decision as to 
the style of the question could be selected from any form 
and should have no effect on the results. It was deter- 
mined, from remarks by some (but not all) students, that 
changes in the statement of the "criticism" problem would 
make it more clear so that the questions used for the third 
group of students were selected for the final search-problems. 
In addition, the meaning of the term "document" was not clear 
to all students — most interpreted the term to mean some 
governmental or official publication— and the word "material" 
was substituted. 

Elapsed time to complete the eight searches was obtained 
for 27 of the 28 participants. Times ranged from a low of 
13 minutes to a high of 63 minutes. A mean time of 31 minutes 
and a standard deviation of eleven minutes was computed. 

Based on these data it was determined that the final exercise 
v 7 ould contain no more than twelve problems requiring an 
average of just less than 47 minutes. The remaining part of 
the hour was planned for explaining the procedure and post- 
test interviews. 

Observation of the procedure and patterns of searching 
seemed to justify the assumption that for specific requests 
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th© appropri cite sections of the catalog could be deterinined. 

As was expected* there were instances when the participant 
selected to search the catalog for less specific subjects 
than the ones presented. Post-test interviews revealed 
that rewording the subject request would in no way have 
made it clearer to the participant that such a subject actually 
was used in the catalog. There was no reason* however* to 
suspect that this same problem (another cause for failure) 
would not occur with the two final test groups nor to expect 
that this would occur more frequently with the users of one 
type of catalog than the other. Of particular interest was 
the relatively consistent pattern of search whether it was 
for the precise subject heading requested or for a less 
specific term. 

One difficulty led to a major change in the final 
selection of the search— problems . Most students participat- 
ing in the pre-test appeared to conclude that the search was 
sufficiently complete when they had discovered some cards 
appropriate to the question. In those cases in which the 
requested subject (a person or corporate entry) was not 
represented by subject or non— subject added entries, there 
was some reluctance on the part of the student to say there 
are none" and many began to search under other headings. To 
avoid this difficulty* those questions were excluded from the 
final set of potentially useful problems. This reduced the 
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final coiuparison between the two catalogs to nine conventional 
subject headings I 26 personal authors, two critic isitis and 
five corporate entries as subject headings. 

As a further result of the pre— test, it was discovered 
that the class standing was not a sufficient basis for in- 
dicating potential experience and familiarity with the main 
catalog. This was due, in many cases, to the number of 
graduate students who had enrolled at Illinois after com- 
pleting their undergraduate work at other schools . While 
the final test group did not include graduate students, the 
possibility of drawing transfer students in the sample could 
result in similar difficulties. The General Information 
Form was revised to include a request for the total number of 
semesters each student had spent on the campus of his respec- 
tive institution. 



Final Comparison 

The final comparison of the two catalogs was conducted 
two weeks prior to the commencement of the testing. Three 
conventional subject headings, eleven personal names as sub- 
jects, one criticism and three corporate entries were judged 
to be similar enough for the purposes of this study. To 
test the assumption that known-item searches would not be 
affected by the arrangement of the catalog, two questions 
were prepared with information about the author and title. 
The remaining ten questions were selected from the 18 com- 
parable sections of subject cards. All three conventional 
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subjects, 
cism, were 



and three corporate entries as well as the criti 
selected. A final decision remained to select 



three of the eleven author headings. One, Jackson Pollock, 
was identical in the two catalogs and was included. The 



second, Otto Ludwig, was selected because of the language- 
requiring precise identification of the meaning of the sub- 
ject heading and reducing the effect of other bibliographical 
information on the card. The third personal author was 
selected to represent a compound name to see what search 
patterns would develop. The entry selected was for Mirabeau 
for whom the established entry is; Mirabeau, Honore Gabriel 
Riquetti, comte de. During the preliminary comparison of the 
two catalogs it was discovered that cross references from the 
unused to the used form for an entry were to be found, at 
Wisconsin, in the author-title catalog only. To investigate 
the effect of this procedure on the use of the subject cata- 
log it was decided to vary the request for material about 
Mirabeau. Thus, some students received the request to find 
material about (rather than by) "Honore Gabriel Riquetti" 
and others for "Honore Gabriel Riquetti (Count Mirabeau)." 
Since this did not represent the same search for all in- 
dividuals, the results were not included in the computation 
of the mean success score. Hence, the success score would 
be computed by taking the mean for the scores achieved for 
the single criticism, the three conventional subjects, two 
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persons— as-subjects , and either one or three of the corporate 
entries as subjects (depending on the test to evaluate the 
effect of the difference in alphabetical arrangement between 
the two catalogs) • 

The final group of questions, and alternatives, is 
given in Appendix B. An analysis of the final search- 
problems and the effects of slight differences in the file 
sizes on the computation of a meaningful success score are 
discussed in detail in the succeeding chapters. 

Selection of the Participants 

The selection of participants was ' achieved by random 
selection from the undergraduate population of the two uni- 
versities chosen for the study. Since lists of the student 
population by classes could not be secured, the student 
directories of the respective schools were used. Random 
selection was used in order to meet one of the underlying 
assumptions for statistical test — random selection from a 
normally distributed population — and to reduce any unknown 
bias in response rate that may result from other selection 
techniques. The anticipated statistical tests also dictated 
the desirability of having at least 30 matched pairs. Based 
on a predicted rate of undeliverable mail (10 percent) and a 
predicted rate of response (30 percent) it was decided to 
select 200 names from the Wisconsin directory and 300 names 
from the Illinois directory. 
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The Illinois directory contains 97 entries per page on 
298 pages plus 46 entries on the final page, a total of 
28,952 names. This figure is slightly smaller than the total 
number of registered students as given in Table 1. The 
Wisconsin directory contains a varying number of entries per 
page with three columns to the page. A sample of 25 columns 
indicated that there was an average of 72 names per column 
with some columns having as few as 68 and as many as 78 
entries. Random numbers for page and entry for Illinois 
and page, entry and column for Wisconsin were selected and 
combined. Combinations of digits that yielded entries out- 
side the universe (graduate students or other unclassified 
students) were excluded. This procedure yielded 216 names 
for Wisconsin students and 331 Illinois students. 

Letters requesting the participation of Illinois and 
Wisconsin students were mailed to their campus addresses so 
as to arrive during the first day of the second semester of 
the 1966-67 academic year. Wisconsin students indicated 
willingness to participate by returning a post card; 

Illinois students were asked to complete and return the 
General Information Form (see Appendix C) . This form was 
completed later by the Wisconsin students. A summary of 
the response rate is given in Table 4. 

Upon arrival on the Madison campus, the investigator 
called each student and scheduled him for a one-hour period 
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TABLE 4 



RESPONSE RATE TO LETTERS REQUESTING PARTICIPATION 





Wisconsin 


Illinois 


Number of letters sent 


216 




331 




Undelivered (Returned) 


9 




11 




Total delivered letters 


207 


(100%) 


320 


(100%) 


Responses 










Willing to participate 


81 


(39.1) 


146 


(45.6) 


Unable to participate 


5 


(2.4) 


9 


(2.8) 


No response 


121 


(58.5) 


165 


(51.6) 



during th© second or third week of classes. The hour selected 
was arranged so that it did not coincide with class periods, 
thus assuring the investigator a full hour with each student 
without concern over class conflicts. The need to conduct 
the test during all hours of the day decreased the oppor- 
tunities for locating and scheduling Wisconsin participants 
to a greater degree than at Illinois (where assistants were 
available) . An error in scheduling during the seventh day of 
the Wisconsin phase resulted in two students reporting at the 
same time. Under the expedience of necessity it was found 
to be no hardship to have two students on the floor at the 
same time. This field-tested change proved to be a neces- 
sity during the latter days of the Illinois phase. As a 
result it was possible to complete 56 hours of floor time 
in 54 hours at Wisconsin and 115 hours of floor time in 81 
hours at Illinois. The Illinois phase of the study, dealing 
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with a larger group of students, was begun during the third 
week of classes and completed in the seventh week. 

Summary 

The objective of the study was to determine if arrange- 
ment of subject entries into a separate file would result in 
a significant change in the effective use of a library cata- 
log. The design of the study was developed in terms of the 
variables that must be considered in an experimental approach 
to testing the hypothesis — the question, the catalog and the 
user. A set of search-problems was developed as a data 
collecting instrument and a formula for scoring the results 
was devised. Two catalogs — one in dictionary arrangement, 
the other divided into subject and non-subject entries — were 
selected and the appropriate sections verified in regard to 
similarity in size and complexity. Undergraduate students 
from the two universities were randomly selected and letters 
were sent inviting them to participate in the study. 

In the process of developing the design for collecting 
data, a number of assumptions were enumerated. This led to 
a second, and equally important, objective of the study: to 

utilize the data secured to explore the appropriateness of 
the matching procedure and the validity of the test instrument 






CHAPTER III 

COLLECTION OF THE DATA 

The purpose of this chapter is to describe the pro- 
cedure used in securing the test data as well as the method 
and results of matching Illinois and Wisconsin participants. 
Two problems connected with the test instrument are also 
discussed. Succeeding chapters will describe the testing of 
the major hypothesis and the investigation of the validity 
of some of the underlying assumptions. 

The Test Groups 

The letters requesting students to participate in this 
study resulted in 81 affirmative responses from the students 
contacted at Wisconsin and 146 from Illinois. Upon arrival 
of the investigator on the Madison campus, students were 
called and appointments were arranged. Some difficulty in 
reaching a few of the respondents resulted in further re- 
duction in the number of students actually included in the 
study. In addition, a few students indicated that the avail- 
able hours were inconvenient or had reconsidered the demands 
of their course work and requested that they be withdrawn from 
the test group. Similar problems in scheduling or contacting 
students by telephone also reduced the number of final par- 
ticipants at Illinois. Since it was more important, for 
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purposes of matching convenience, a greater effort was made 
to contact all students at Illinois. Seventeen students who 
could not be reached by phone were sent a post card request- 
ing them to call for an appointment. Table 5 is a summary 
of the actual number of participants for each school. 



TABLE 5 

TOTAL NUMBER OP PARTICIPANTS IN THE STUDY 





' Wisconsin 


Illinois 


Total affirmative responses 


81 


146 


Requested to be withdrawn 


7 


11 


Unable to contact 


16 


10 


Contacted and scheduled 


58 


125 


Scheduled but did not appear 


2 


10 


Produced unusable data 


6 


21 


Pinal test group 


50 


94 



Of the total 171 students who took part in the study, 
five Wisconsin students and 18 Illinois students were unable 
to complete the search-problem exercises within the alloted 
period of time. In addition, four students (one at Wisconsin 
and three at Illinois) who were listed as underclassmen in 
their respective directories were found to be enrolled in 
professional programs and in their fifth or sixth year of 
college. These students were considered to be outside of the 
intended student universe and were deleted from the study. 

The final number of usable scores was 50 students at Wisconsin 
and 94 at Illinois, a total of 144 participants. The 
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distribution of participants by sex and class is given in the 
following table: 



TABLE 6 

CHARACTERISTICS OF THE FINAL TEST GROUPS 



Class 


Wisconsin 






Illinois 




Female 


Male 


Total 


Female 


Male 


Total 


Freshmen 


12 


3 


15 


12 


22 


34 


Sophomores 


2 


4 


6 


10 


12 


22 


Juniors 


10 


9 


19 


8 


11 


19 


Seniors 


2 


8 


10 


7 


12 


19 


Totals 


26 


24 


50 


37 


57 


94 



Data Collection Procedure 

The procedure in conducting the search exercises was 
similar at the two schools with two exceptions. The Wisconsin 
students were required to complete the General Information 
Form at the time they appeared for their appointments. Illinois 
students, who had completed the Form as an indication of willing- 
ness to participate, were asked only to clarify or explain any 
responses which were not clear. Wisconsin students reported 
to the main public catalog room in the University library 
where the investigator was located at a large study table at 
the rear of the room. Instructions, interviews and the exer- 
cises were all conducted in that room. At Illinois, students 
reported to the investigator's office, located on the fourth 
^ floor of the main library building, and were escorted to the 

o 
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main catalog (on the second floor) after the instructions 
were given. Some of the instructions and post-test inter- 
viewing took place while going to or returning from the 
catalog. This small variation may explain the slightly 
higher rate of uncompleted exercises at Illinois (12.3 per- 
cent) as compared to Wisconsin (8.9 percent). 

Students were informed of the purpose of the exercise 
and the procedure to be used.^ The participant was then in- 
structed to complete the searches, indicating his answers to 
the investigator rather than writing his response on the card 
containing the question. Questions were presented to the par 
ticipant on 4" x 6" cards as illustrated in Figure 1. All 



FIGURE 1: SEARCH-PROBLEM FORMAT 



jocate the appropriate catalog cards that 
the library contains material about the topic. 



Statistical design 

( ) None in the university catalog. 

( ) Cards located in drawer (s) number: 



Search No. : 



^See Appendix D for "Investigator 



s .Interview Schedule." 
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exercises were then completed with the investigator present. 

In those cases when there were two students being tested at 
the same time, each student was instructed to wait for the 
investigator to note the answer before proceeding to the 
next problem. 

The arrangement of the catalogs at Wisconsin made it 
possible to keep the students in sight and to note their pro- 
cedure. These notes were later verified as the participants 
were asked to verbalize their procedure after reaching a con- 
clusion. At Illinois, the presence of the public shelf list 
in the center of the floor occa,sionally obscured some students. 
By noting what question each participant was searching, it 
was possible to be located in such a position, as to minimize 
this physical obstruction. V/henever this was not possible, 
each step of the search strategy was verified with the student. 
There was no apparent indication that this modification in 
procedure in any way invalidated the data collected; it may, 
in fact, have reduced any upsetting effects of the investi- 
gator constantly being present during the search. 

In support of this contention, the success scores for 
the Illinois group were used to test the hypothesis that there 
was no significant difference between the group means of those 
who conducted the searches alone and the students who were 
tested two at a time. While the two groups were not randomly 
selected, an underlying assumption of most statistical tests, 
the scheduling of individuals in hours convenient to the 
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examiner and the participant, the failure of some students 
to appear and the arrangement of appointments by three dif- 
ferent people would seem to have decreased any systematic 
bias in the assignment of hours. 

Statistical measures for the participants who performed 
alone (Group I) and for those who performed in pairs (Group II) , 
given in Table 7, were computed and no difference between the 
means was observed. 



TABLE 7 





STATISTICAL MEASURES 


FOR THE TWO 


ILLINOIS 


GROUPS 




Group 


N 


X 


x2 


X 


S2 


S 


I 


42 


18.013 


8.612 


0.429 


0.021 


0.145 


II 


52 


22.291 


11.168 


0.429 


0.031 


0.176 



As each participant searched the catalog to find the ap- 
propriate cards, the investigator noted the procedure as well 
as the final decision. Every effort was made to secure infor- 
mation about where the individual searched, what specific head- 
ing he or she had in mind, the type of problem that might be 
encountered, etc. For example, one participant's response for 
the problem shown in Figure 1 was: 

Participant (search number 023) first searched 
under the heading "Design, Statistical" in the ap- 
propriate sequence of the alphabet, found nothing 
and continued to scan through drawer; 






wm 



54 

Second search was under "Statistical," found 
a cross-reference under "Statistical design" and 
went to: 

"Experimental Design" ; indicated all cards 
with the subject added entry as appropriate. 

During the instruction period students were informed 
that the test questions had been selected randomly and that 
some of the requests might not represent areas of interest to 
them. It was assumed, however, that the student could cope 
with the request at a level determined by his basic knowledge 
about the catalog.. At the termination of each session, a 
post-test interview was conducted. One of the questions 
asked of the participant was "Did you find that the problems 
and your responses were a fair indicator of your general 
knowledge of the scope and arrangement of the catalog?" 

The replies of the students, admittedly testimonial, in- 
dicated no reason to suspect the validity of this approach 
to measuring effective use. Some students, however, were 
troubled by the abstraction of the problems, saying that they 
wouldn't start to look for answers to some of the questions 
in a catalog but would prefer, rather, to use some reference 
tool or search the periodical literature. This criticism 
can be considered to be a legitimate objection to the spe- 
cific questions rather than an indictment of the procedure 
and must be considered a limitation to the study. 
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Matching Illinois and Wisconsin Students 
While the Wisconsin phase was being completed, Illinois 
students had returned. the completed General Information Form 
as an indication of their willingness to participate. The 
purpose of using the Form rather than a post card reply was 
to permit matching students prior to scheduling. This pro- 
cedure was abandoned when it became clear that not all stu- 
dents might complete the exercise or that some students would 
change their minds about participating or forget their appoint- 
ments. The Illinois participants, therefore, were scheduled 
and completed the search-exercises prior to matching. 

In very general terms, the matching was successful to 
the extent that 31 pairings were made. While there were 
some differences between the matched pairs, every effort was 
made to have the two primary criteria ("semesters on campus" 
and "frequency of use of the main catalog") as equal as 
possible. To facilitate comparison, the responses to the 
question of frequency of use of the catalog have been con- 
verted to a code score as given in Table 8. 

One criterion for which information was secured on the 
General Information Form was not used in the matching pro- 
cedure. In the course of conducting the pre-test interview 
it was determined that the responses to the question on the 
"frequency of use of departmental catalogs" were not com- 
parable. Most Wisconsin students interpreted this question 
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TABLE 8 



CODE FOR "FREQUENCY OF CATALOG USE" RESPONSE 



Freauency of Use 


Code 


Less than once a semester 


1 


1 or 2 times a semester 


2 


3 to 5 times a semester 


3 


6 to 10 times a semester 


4 


11 to 15 times a semester 


5 


16 or more times a semester 


6 



to mean the Wisconsin reserve book catalog. The catalog, 
which is on key-punched cards, is sufficiently different in 
format and content to make it meaningless to compare it with 
the more conventional departmental catalogs located at the 
two schools. 

Basic information about the participant, as listed in 
Table 9, was then typed on a card and these were separated 
into groups according to the number of semesters on campus. 
These groups were subdivided into categories according to 
the frequency of the use; of the main catalog and subsequent 
matching was accomplished by scanning other characteristics. 
(This information plus the success ratio for each question 
was later entered on machine-readable cards for computer 
analysis.) 
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TABLE 9 

CHARACTERISTICS OF MATCHED PAIRS OF STUDENTS* 



Student 

Identi- 

fication 


Semes- 
ters on 
Campus 


Cata- 

log 

Use 


Sex 


Class 


GPA 


Mai or 


006 


3 


2 


M 


4 


2.7 


Political Sci. 


461 


3 


2 


M 


2 


3.9 


Economics 


007 


1 


3 


M 


3 


3.5 


Civil Engr. 


507 


1 


3 


M 


1 


3.8 


Mech. Engr. 


008 


7 


2 


M 


4 


3.3 


Economics 


513 


7 


2 


M 


4 


3.0 


Psychology 


009 


3 


1 


M 


2 


3.2 


English 


488 


3 


1 


F 


2 


2.8 


English 


Oil 


1 


2 


F 


3 


3.5 


Social Work 


510 


1 


2 


M 


3 


4.0 


Sociology 


012 


3 


1 


M 


2 


3.1 


Business 


478 


3 


1 


M 


2 


3.1 


Finance 


013 


1 


2 


M 


3 


3.0 


Chemistry 


532 


1 


2 


M 


1 


2.3 


Pre-Dentistry 


014 


1 


J. 


M 


1 


3.2 


Pre-Commerce 


^ 487 


1 


1 


M 


1 


3.0 


Accounting 


015 


1 


T_ 


F 


1 


3.1 


Nursing 


504 


1 


i 


F 


1 


2.1 


Nursing 


016 


3 


4 


M 


2 


3.1 


Sociology 


473 


3 


4 


M 


2 


3.3 


Administration 


019 


1 


4 


F 


1 


4.8 


French 


502 


1 


4 


F 


1 


4.5 


Latin Am. Studies 


020 


5 


6 


F 


3 


4.1 


Psychology 


531 


5 


5 


F 


3 


3.9 


History 


021 


3 


4 


F 


3 


3.8 


History 


498 


3 


4 


F 


3 


3.6 


English 


023 


3 


2 


F 


3 


4.1 


Zoology 


466 


3 


2 


F 


2 


4.0 


Home Economics 


029 


1 


1 


M 


3 


3.9 


Education 


457 


1 


1 


M 


1 


3.7 


Psychology 


030 


7 


4 


M 


4 


4.3 


History 


471 


7 


4 


M 


4 


4.5 


Anthropology 


032 


1 


1 


F 


1 


3.8 


Liberal Arts 


528 


1 


1 


M 


1 


3.7 


Education 


035 


5 


1 


M 


3 


3.6 


Chemistry 


523 


5 


1 


M 


3 


3.3 


Mech^ Engr. 
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TABLE 9 (continued) 



Student 

Identi- 

fication 


Semes- 
ters on 
Campus 


Cata- 

log 

Use 


Sex 


Class 


GPA 


Manor 




1 


1 


F 


1 


3.6 


Medical Tech. 


Vi/ U 

516 


JL 


1 


M 


1 


3.5 


Pre-Dentistry 


037 


7 


1 


M 


4 


3.6 


Civil Engr. 


\J i 

557 


7 


1 


M 


4 


3.5 


General Engr. 


038 


5 


3 


M 


3 


3.6 


Economics 


567 


5 


3 


F 


3 


3.5 


Retailing 


030 


6 


4 


F 


4 


3.1 


Social Work 


\j -j j 

505 


6 


4 


F 


4 


3.4 


Social Studies 


042 


1 


1 


F 


1 


3.7 


French 


501 


1 


1 


F 


3 


3.5 


Art Education 


045 


1 


2 


F 


1 


3.6 


Art Education 


526 


1 


2 


F 


1 


3.7 


Elem. Education 


047 


1 


4 


F 


2 


4.6 


English 


45?- 


1 


3 


F 


2 


4.7 


English 


048 


1 


1 


M 


1 


3.6 


Chemistry 


546 


1 


1 


M 


1 


3.5 


Electrical Engr. 


050 


7 


5 


F 


4 


3.1 


English 


459 


7 


4 


F 


4 


3.4 


English 


055 


1 


1 


M 


1 


2.0 


Agric. Engr. 


462 


1 


1 


M 


1 


2.5 


Aeron . Engr . 


056 


3 


1 


F 


2 


3.7 


History 


w 

519 


3 


1 


F 


2 


3.8 


Elem. Educ. 




1 


3 


F 


3 


4.2 


Political Sci. 


U30 

493 


1 


3 


F 


3 


4.2 


Speech Educ. 


060 


7 


1 


M 


4 


3.5 


English ^ 


495 


7 


1 


F 


4 


3.4 


Anthropology 

- a ■ • 



*The coaes ror i:ne vai. 

are as follows: For "catalog use" see code as given in 

Table 8; for "class," 1 = Freshmen, 2 = Sophomores, 

3 = Juniors and 4 = Seniors; the Grade Point Average (GPA) 
is based on a five-point system (A = 5.0). 
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Computation of the M ean Success Scores, 

All responses to the test questions were scored using 
the success ratio formula (see p. 28) . The success ratios 
were then used to compute a mean success score for each in- 
dividual. As noted previously, a number of factors had to be 
considered in determining which of the questions would be in- 
cluded in the final success score. One of these was the matter 
of alphabetical arrangement of subordinate agencies (as out- 
lined in Chapter II) . A second factor was the slight differ- 
ence in some of the card files at Wisconsin and Illinois. 

The effect of differences in filing, . — As indicated, the 
success ratio for searches for which the appropriate entry is 
a subordinate agency may be affected by the differences in 
file organization as well as by the possible confusion between 
subject added entries and non-subject added entries. To test 
the effect of filing, the two questions which were directly 
affected were analyzed. The problems, as presented to the 

students, were stated as follows: 

Locate the appropriate catalog cards that 
indicate the library contains material about 
(rather than by) : The U.S. Civil Service Com- 

mission. 

Locate the appropriate catalog cards that 
indicate the library contains material about 
(rather than by) : The Great Britain Board of 

Trade . 

The analysis was based on an examination of the pro- 
cedure followed by students at Wisconsin and Illinois. Two 
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general alternatives were considered: 1) the patron selected 

the appropriate term to search, or 2) the patron selected 
some other term to search (e.g., under "Civil Service" 
rather than "U.S. Civil Service Commission"). Frequencies 
for those who chose the correct search term were tabulated. 

The category "found" includes all students who located the 
term even if the student made some subsequent error in selec- 
tion of cards. The expected and observed frequencies (in 
parentheses) for the two questions are given in Tables 10 and 11 



TABLE 10 

OBSERVED AND EXPECTED FREQUENCIES FOR 
"U.S. CIVIL SERVICE COMMISSION" PROBLEM 



Catalog 


Found 


Did Not 
Find 


Total 

(Rows) 


Wisconsin 


8 

(13.18) 


7 

(1.82) 


15 


Illinois 


50 

(44.82) 


1 

(6.18) 


51 


(Total: 

Columns) 


58 


8 


66 



The calculated value of chi-square for the measures 
given in Table 10 is 21.72, which is significant at the .05 
level for one degree of freedom. Therefore, it was deter- 
mined that the difference in filing actually did have an 
effect on the possible success or failure in searching for 
this problem and the score was deleted from the computation 



of the mean success score. 



TABLE 11 



OBSERVED AND EXPECTED FREQUENCIES FOR 



"GREAT BRITAIN 


BOARD OF 


TRADE" 


SEARCH 




Did Not 


Total 


Catalog 


Found 


Find 


(Rows) 


Wisconsin 


21 


10 


31 




(18.82) 


(12.18) 




Illinois 


30 


23 


53 




(32.18) 


(20.82) 




(Total: 








Columns) 


51 


33 


84 



The calculated value for chi-square for the frequencies 
given in Table 11, which is 1.02, is not significant at the 
.05 level for one degree of freedom. For this reason, the 
"Great Britain" question was not excluded from the computa- 
tion of the mean success score. The curious difference of 
the effect of the filing on the two problems is examined in 
some detail in Chapter V. 

The final mean success score, therefore, was an average 
of the success ratio for three conventional subject searches, 
one criticism, two corporate entries as subject headings and 
two personal entries as subject headings, a total of eight 
searches . 

The effect of file size differences . — The file size of 
the subject portions of the two catalogs should be identical 
in order to ensure similar results for identical procedures. 
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The use of existing catalogs at two different institutions, 
however, required that some latitude be allowed. While the 
file size of some entries such as "Jackson Pollock" and 
"Amateur Athletic Union" were identical (three and one 
entries, respectively) others, such as "Experimental Design" 
varied (58 at Illinois, 52 at Wisconsin). These small dif- 
ferences were considered to be less serious (and nothing in 
the test procedure contradicted this assumption) than dif- 
ferences in two of the other files. The problem was one in 
which the confusing (or inappropriate) entries were contained 
within the subject files. 

The "Otto Ludwig" file at Wisconsin contained 29 sub- 
ject added entries and an additional eight criticisms of 
various Ludwig titles. Illinois' file, on the other hand, 
contained 34 subject added entries and ten criticisms. 
Technically (and for the purposes of this study, accurately) , 
the criticisms are not appropriate responses to the question 
asked. The individual who may select "all the red ones" at 
Illinois — which happened quite frequently — or all entries in 
the subject catalog at Wisconsin without discrimination, would 
be given the following scores : 



Wisconsin: 


2 

3 = = 


(29) (29) _ g . 

(29) (37) 


Illinois: 


S = _ 

kt 


(34) (34) _ Q 7-70 
(34) (44) 
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For identical searches and decisions, the Illinois student 
would receive only 98.6 percent of the Wisconsin student's 
score; an adjustment by this figure to the Wisconsin score 
should be made. 

Similarly, the total "Firearms" file at Wisconsin 
contained 55 entries, of which two are the appropriate 
"Firearms — Identification" entries. The Illinois file con- 
tains 94 cards of which five contained the appropriate head- 
ing and three were title added entries. Again, numerous stu 
dents at Illinois and Wisconsin indicated that they believed 
all entries under Firearms (and its various subdivisions) 
would be the appropriate cards. This decision must be con- 
sidered partially correct since the appropriate cards are 
contained within the indicated deck. Computing success 
ratios for the two catalogs yields a score of S = 0.036 at 
Wisconsin and S = 0.055 at Illinois (deleting the conflict- 
ing title added entries) . Thus, the Wisconsin score yields 
only 65.5 percent of the value at Illinois, although these 
are identical searches; the Wisconsin score should be 
corrected by a factor of approximately 1.53. 

When the correction factors are used to create new sue 
cess scores, the difference is relatively small. Table 12 
gives the mean success scores computed with the original 
uncorrected method (designated the "raw" score) and the 
values using the correction factor for eight individuals 
affected by both corrections. 
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TABLE 12 



COMPARISON OF EIGHT WISCONSIN MEAN SUCCESS SCORES WITH 
CORRECTION FACTOR FOR "FIREARMS" AND "LUDWIG" SEARCHES 



Student 

Number 


Raw 

Score 


Corrected 

Score 


Student 

Number 


Raw 

Score 


Corrected 

Score 


013 


.353 


.354 


037 


.603 


.604 


018 


.348 


.351 


040 


.103 


.104 


022 


.478 


.479 


041 


.130 


.132 


030 


.728 


.729 


060 


.228 


.229 



While the difference between these scores for each in- 
dividual is statistically insignificant (using a ^-test for 
difference between means) it does demonstrate the potential 
cumulative effect upon the total success score and the pos- 
sible bias that would result from all files being slightly 
different in favor of one catalog. 

Mean success scores for the 31 pairs of students were 
calculated without using the correction factor. The results 
of the test of the hypothesis is described in the following 
chapter . 

Summary 

Usable data from 50 Wisconsin and 94 Illinois students 
was secured over a six-week period during the second semester 
of the 1966-67 academic year. Information about various 
personal characteristics and catalog use habits was gathered 
from all participants. Based on this information, 31 pairs 
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of Wisconsin-Illinois students were designated and the mean 
success scores of these individuals were used to test the 
major hypothesis* 

Due to some peculiarities in the search procedure 
associated with a difference in filing rules, one question 
was deleted from the computation of the mean success score. 

For the purpose of testing the hypothesis, the success score 
of the final test group represented an average of each 
student's scores on eight questions: three conventional 

subject searches, one criticism, two corporate entries as 
subject headings and two personal entries as subject headings. 
The results of the test of the hypothesis and the investiga- 
tion of some of the underlying assumptions are described in 
Chapter IV. 



CHAPTER IV 



ANALYSIS OF THE DATA 

This chapter contains the results of the test of the 
major hypothesis. In addition, the effects of the arrange- 
ment of the catalog on known-item searches is analyzed. A 
third objective is the analysis of a number of assumptions 
enumerated in previous chapters upon which the validity of 

tVi© invsstigs.t iv© proc©d.ur'© is bas©d.« 

Th© und©r lying assumptions can b© class if i©d according 

to th©ir ©ff©ct on thr©© asp©cts of th© study. On© assump- 
tion conc©rns th© appropriat©n©ss of th© crit©ria s©l©ct©d 
for th© matching of th© r©spond©nts. S©cond, th©r© ar© 
G©rtain assumptions r©lating to th© construction and validity 
of th© t©st instrum©nt. Data p©rtin©nt to th©s© two assump- 
tions ar© analyz©d in this chapt©r. In th© following chapt©r, 
a third assumption conc©rning th© pr©dict©d similarity of 
patterns of search, using either catalog, is explored in 
detail as part of th© general description of catalog search 

procedures . 

Testing of the Hypothesis 

Mean success scores, based on eight searches, were 
calculated for the 31 pairs of students. The original 
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hypothesis to be tested was: 

Assuming all other factors are equal, sub- 
ject searches through a catalog in which the 
subject entries have been separated (e.g., a 
divided catalog) will produce more pertinent 
references and less inappropriate references 
than identical searches using a file combining 
all entries into a single (dictionary) sequence. 

Given a mean success score which represents the number of 

pertinent and/or inappropriate references retrieved for a 

specific subject search, the hypothesis can be restated as 

follows : 

Assuming all other factors are equal, the 
mean success score for an individual using a 
divided catalog will be significantly greater 
than the resulting score for the same searches 
using a dictionary catalog. 

Testing the null hypothesis (i.e., no difference between 
groups) , a ^-test for difference between means was used to 
determine the statistical significance of the experimental 
data. 

A summary of the data for the 31 pairs is given in 
Table 13. (Complete calculations of the jb-test are given in 
Appendix E.) For 30 degrees of freedom (N-1) , the expected 



value of the t statistic at the .05 level for a one-tailed 
I test is 1.697. That is, a value of t calculated from the. 

I test group can be expected to be 1.697, or less, by chance 

I alone . 
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1 



68 



TABLE 13 



DATA 


FOR THE 


31 MATCHED 


PAIRS 




Pairs 

Wis. - 111. 


Wis . 
Scores 


111. 

Scores 


(X 1 --X 2 ) 

D 


(Xj^-X2)^ 

d2 


006 - 461 


.723 


.298 


.425 


.1806 


007 - 507 


.723 


.381 


.342 


.1169 


008 - 513 


.598 


.500 


.098 


.0096 


009 - 488 


.348 


.266 


.082 


.0067 


Oil - 510 


.356 


.294 


.062 


.0038 


012 - 478 


.473 


.453 


.020 


.0004 


013 - 532 


.353 


.111 


.242 


. 0585 


014 - 487 


.147 


.315 


-.168 


.0282 


015 - 504 


.125 


.467 


-.342 


.1169 


016 - 473 


.875 


.667 


.208 


.0432 


019 - 502 


.360 


.772 


-.412 


.1697 


020 - 531 


.375 


.480 


-.105 


.0110 


021 - 498 


.723 


.347 


.376 


.1413 


023 - 466 


.598 


.401 


.197 


.0388 


029 - 457 


.473 


.442 


.031 


.0009 


030 - 471 


.728 


.580 


.148 


.0219 


032 - 528 


.598 


.339 


.259 


.0670 


035 - 523 


.723 


.621 


.102 


.0104 


036 - 516 


.371 


.314 


.057 


.0032 


037 - 557 


.603 


.728 


-.125 


.0156 


038 - 567 


.348 


.298 


.050 


.0025 


039 - 505 


.996 


.474 


.522 


.2724 


042 - 501 


.598 


.478 


.120 


.0144 


045 - 526 


.598 


.480 


.118 


.0139 


047 - 451 


.683 


.612 


.071 


.0050 


048 - 546 


.723 


.343 


.380 


.1444 


050 - 459 


.223 


.250 


-.027 


.0007 


055 - 462 


.250 


.267 


-.017 


.0002 


056 - 519 


.598 


.450 


.148 


.0219 


058 - 493 


.000 


.529 


-.529 


.2798 


060 - 495 


.228 


.371 


-.143 


.0204 
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The experimental data were tested for significance 
using the following formula: 



From the evidence of the experimental group, there is no 
reason to reject the null hypothesis that there is no dif- 
ference between the two groups. Assuming the validity of the 
underlying assumptions appropriate to the procedure for 
matching and testing, the divided catalog did not result in 
a more effective catalog for the two groups. Since confusion 
about the differences between similar headings used to desig- 
nate subject and non-subject entries was observable during 
the study (as discussed in Chapter V) , it must be concluded 
chat this cause for failure is not a significant contributor 
to the predictable level of effective use of a catalog. 

It is only natural, then, to inquire if the arrangement 
of the catalog does not increase or decrease the effective 
use of the catalog in making subject searches, does arrange- 
ment have an effect on known-item searches? In order to col- 
lect evidence to support the hypothesis that difference in 
arrangement does not affect known- item searches, two questions 
were included in the search-problem exercises that were non- 
subject searches. Table 14 i^s a summary of the observed and 



t 




501 - .430 
.0423 



1.678 
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TABLE 14 

OBSERVED AND EXPECTED FREQUENCIES IN 
SUCCESSFULLY LOCATING THE CALL NUMBER 
FOR THE "NAGEL" QUESTION 



Catalog Used 


Found 


Did Not 
Find 


Total 
(Rows ) 


Wisconsin 


50 


1 


51 




(50) 


(1) 




Illinois 


101 


2 


103 




(101) 


(2) 




(Total: 








Columns) 


151 


3 


154 



expected frequencies of success and failure in locating the 
call number for the book by Ernest Nagel. The table includes 
the responses of all students completing this problem. 

Since there is no difference between the expected and 
the observed frequencies the value of the statistic chi-square 
is zero. There is no reason, therefore, to suspect any re- 
lationship between the frequency of success and the arrange- 
ment of the catalog. 

For the frequencies given in Table 15, the computed 
chi-square also was not significant (chi-square = 0.6). 
Therefore, there is no evidence to indicate a significant 
difference in successfully locating known— items in one type 
of catalog or another. 

Conclus ions . ——Based on the evidence of the experi- 
mental study described, there is no reason to suspect that 
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P TABLE 15 



OBSERVED AND EXPECTED FREQUENCIES 
IN SUCCESSFULLY LOCATING THE CALL NUMBER 
FOR THE "WARREN COMMISSION REPORT" QUESTION 



Catalog Used 


Found 


Did Not 
Find 


Total 

(Rows) 


Wisconsin 


33 

(35.2) 


21 

(18.8) 


54 


Illinois 


70 

(67.8) 


34 

(36.2) 


104 


(Total: 


Columns) 


103 


55 


158 



dividing a catalog is a satisfactory device for making the 
subject approach more effective — inasmuch as it did not result 
in significantly reducing the failure attributed to the con- 
fusing of non-subject entries with subject entries. At the 
same time, there is no evidence to support the belief that 
dividing the catalog would actually reduce effective use 
either in respect to subject searches or known-item searches. 
These conclusions, however, must be considered tentative for 
two reasons. First, the study dealt with students attending 
two large mid-west universities and in no way can inferences 
be made to divided and dictionary catalogs in general, with 
any degree of confidence, without replication of the study. 
Second, the validity of the underlying assumptions of the 
study is subject to objective verification. The remainder of 
this chapter contains a description of the analysis undertaken 
to substantiate the appropriateness of some of these assump- 



tions . 
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Validation of the Matchi ng Procedure 
The Hatching of the participants was predicated on the 
assumption that frequency of use and the number of semesters 
on campus constituted the best criteria for equating in- 
dividuals. This assumption was considered most appropriate 
when note is made of one of the conclusions in the Jackson 
study that "The patron's inexperience and unfamiliarity with 
the catalog was another principal source of difficulty. 



In order to compare these two primary criteria with other 
potentially useful characteristics for matching, a series 
of questions was asked of the participants on the General 
Information Form (Appendix C) . Data on the following charac 



teristics were collected: 1) semesters on campus and class 



standing (also converted to semesters in college) as measures 
of exposure to the catalog; 2) frequency of the use of the 
main catalog as a measure of familiarity; 3) sex; 4) cumu- 
lative grade point average; 5) most common approach to using 
the catalog; 6) the type and amount of instruction received 
in "how to use the library"; and, 7) work experience in 

libraries. 

The proce(lure for analyzing these data was to investi- 
gate the relationship between personal characteristics and 
the students' performance at each school. This analysis was 
based on the concept that one or more characteristics could 



^Sidney L. Jackson, Catalog Use Study; Director's 

Report, ed. by Vaclav Mosteci^ (Chicago: American Library 

Association, 1958), p. 2. 
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be shown to be related to the mean success score at each 

school. By examining these characteristics it could be 

« 

determined if they were common to the two schools and there- 
fore generally applicable as criteria for matching. Thus, 
for each school, identical guestions (a constant) were 
searched using the same catalog (also a constant) by dif— 
fsrent participants. Hence, the differences in the success 
scores must be attributable to the participant's conduct of 
the search, rather than to the guestion or the catalog. 

Since these comparisons were made within schools, there was 
no reason for not including the "U.S. Civil Service Commis- 
sion" question that was deleted in the comparison of means 

between schools. 

Method of analysis and limitations . — The relationship 
between the various characteristics and performance was deter- 
mined by computing some measure of association where appli- 
cable. For data given in interval measures, a Pearson 

2 

product moment correlation (designated as r) was calculated. 
The resulting correlation coefficients were tested for 

3 

significance using a table of expected values. For data 
v^'hich lent themselves to natural dichotomies (e.g., sex, in- 
struction and work experience) , a point biserial correlation 

^Allen L. Edwards, Statistical Methods for the Be- 
havioral Sciences (New York: Holt, Rinehart and Winston, 

1954), pp. 142-55. 

^William H. Beyer, ed., CRC Handbook of Table s for 
Probability and Statistics (Cleveland: Chemical Rubber Co., 

1966), p. 299. 
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coefficient was computed and tested for statistical sig- 

4 

nificance using a jt-test. Finally, for a few of the 

characteristics, mean success scores were grouped and the 

differences between the means of these groups were tested 

5 

for significance using analysis of variance. 

It must be noted, however, that these analyses must be 
considered tentative at best. The purpose of the random 
selection procedure was to minimize response bias and to 
ensure the best chances for matching. The sample, however, 
cannot be considered — nor was it intended — to be a true proba- 
bility sample of the undergraduate population at either school 
Therefore, the analysis is appropriate to the participants 
only. Even so, this analysis is important if only as a pre- 
liminary investigation in determining any tendency of re- 
lationship between the numerous variables. 

Exposure and familiarity . — A correlation coefficient was 
computed for each school for the variable "semesters in col- 
lege" and "semesters on campus" in respect to the mean success 
score for each group of students. The computed correlation 
coefficients are given in Table 16. (For a summary of 
formulae used and the mean success scores for all students, 
see Appendices F and G.) 

^Helen M. Walker and Joseph Lev, Statistical Inference 
(New York: Henry Holt and Company, 1953), pp. 262-67. 

^William L. Hays, Statistics for Psychologists (New York 
Holt, Rinehart and Winston, 1963), pp. 356-458. 



TABLE 16 

CORRELATION COEFFICIENTS (r) 







Y-Variable 
Mean Success Score 


X-Variable 




Wisconsin 


Illinois 


Semesters in 


college 


.252 


.125 


Semesters on 


campus 


.227 


.162 



The expected correlation coefficients for the Wisconsin 
group (N = 50) was .273 for the .05 level. For Illinois 
(N = 94) f the value of "r" would be between .205 and .195. 
Therefore, none of the correlation coefficients is signifi- 
cant and the reverse relationship and apparent contradition 
between the two schools can be explained as chance occurrence. 

While there was no significant difference when success 
scores were correlated with semesters on campus or in college, 
it was anticipated that there may be some significant dif- 
ference between the means of the various classes. For the 
purpose of analysis, only the larger groups were considered — 
those second semester freshmen, sophomores, juniors and 
seniors who had indicated they had completed one, three, five, 
or seven semesters of college prior to the study. This re- 
sulted in deleting eight students at Illinois and six at 
Wisconsin who fell into intermediate categories. The means, 
of the Illinois group appeared to support the general belief 
that semesters on campus was a better measure than semesters 
in college , as shown in Table 17 . 
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TABLE 17 

MEAN SCORES OP ILLINOIS STUDENTS GROUPED BY SEMESTERS 
IN COLLEGE AND SEMESTERS ON CAMPUS 



Semesters 


College 


Campus 


N 


Mean 


N 


Mean 


1 


35 


.396 


40 


.398 


3 


21 


.425 


24 


.413 


5 


16 


.411 


11 


.428 


7 


14 


.477 


11 


.498 



An analysis of variance yields a value for the F-statistic 
of 0.920 (semesters in college) and 1.202 (semesters on campus) ; 
neither of which is significant at the .05 level. The Wisconsin 
data, summarized in Table 18, did not show the pattern evident 
at Illinois. Similar analysis of the Wisconsin data also 
indicated no significant difference. 



TABLE 18 

MEAN SCORES OF WISCONSIN STUDENTS GROUPED BY SEMESTERS 

IN COLLEGE AND SEMESTERS ON CAMPUS 



Semesters 


College 


Campus 


N 


Mean 


N 


Mean 


1 


15 


.336 


24 


.337 


3 


5 


.574 


7 


.583 


5 


17 


.371 


7 


.400 


7 


7 


.502 


6 


.460 



The evidence of the experimental data indicates that 
there is no reason to consider either variable as superior. 
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In that case, the use of semesters on campus as the criterion 
for matching is as valid, but no better, as semesters in 
college. 

The cumulative grade point average . — In a recent study 
conducted by Snider, grade point average was found to relate 
to test scores on a library ability test administered to 
college freshmen. It was believed that a similar high cor- 
relation may exist between grade point average and the 
students' performance in searching for subject references. 
Correlation coefficients for the 94 Illinois participants and 
the 50 Wisconsin students were calculated. While the result- 
ing correlation coefficient for Wisconsin students (r = .125) 
was not significant, the value for the Illinois group 
(r = .219) was significant at the .05 level. 

This level of association accounts for only 4.8 percent 
of the total variance [(.219) ]. In addition, the general 
relationship of grade point average to success score is not 
completely clear because of the non-significant results of 
the Wisconsin data analysis. Nevertheless, a review of the 
matched pairs was undertaken to determine to what degree dif- 
ferences in grade point average might have altered the match- 
ing. Although the original matching was done first on the 
criteria of semesters on campus and frequency of use of the 
catalog, close attention was paid to matching as many of the 

^Felix Eugene Snider, "The Relationships of Library 
Ability to Performance in College." (Unpublished Ph.D. 
dissertation. University of Illinois, 1965). 









other characteristics as possible. As a result of this pro- 
cedure it was found that 27 pairs had grade point averages 
which were within 0.4 points of each other, an additional 
three pairs were within 0.5 points, while only one pair had 
a difference of as much as one point. 

Frequency of use . — As mentioned, one of the two criteria 
originally selected for establishing matched pairs was the 
frequency of use of the main catalog. Students were asked to 
place themselves in one of six categories ranging from "less 
than once a semester" to "16 or more times a semester." In 
the anticipated analysis of the effect of frequency of use 
on the success score, it was believed that there would be 
some interaction between this criterion and semesters on 
campus (or class standing). Thus, it was considered possible 
that a freshman who used the catalog quite frequently might 
do as well as, or better than, a sophomore or junior who may 
seldom use the catalog. 

Again, the limitation of the small sairiple size and the 
occasion in which just a few individuals represented the 
various combinations of use and semesters on campus, made it 
of questionable value to explore the potential relationship 
for all students. Instead, one of the larger groups of 
Illinois students was selected for analysis. Thus, for a 
fairly large group of participants (Illinois freshmen in this 
case) , it was hypothesized that the mean success score for 
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each of the six categories of frequency of use would be dif- 
ferent and that the mean of the infrequent users would be 
lower than the mean of the next category. For the purposes 
of this analysis, freshmen were defined as those students 
who had been in college and on the Illinois campus for one 
semester • 

The F-statistic computed by analysis of variance 
(F = 2.68) was significant at the .05 level. It must be 
noted, however, that the small number of participants in each 
category casts some suspicion on the results. Some evidence, 
however, to support the contention that the mean success 
score is associated with frequency of use and exposure to 
the catalog was available. When the scores for all students 
were grouped into the six categories without control for 
semesters on carapus (or class standing) , the resulting value 
of F (F = 0.50) was not significant. Table 19 is a summary 
of the group means by frequency of use for the two groups of 
Illinois students. 

Other variables . — Other characteristics for which data 
were collected were thought not to be particularly important 
in the original matching process. These characteristics were: 
1) sex, 2) type and amount of instruction, 3) work experi- 
ence, and 4) approach. The reason for making this assumption 
varied according to the characteristic. During the pre-test 
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TABLE 19 

GROUP MEANS FOR ILLINOIS STUDENTS BY 
"FREQUENCY OF USE" CATEGORY* 
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Frequency 
of Use 


Freshmen 


All Students 


N 


Mean 


N 


Mean 


1 


14 


.352 


28 


.416 


2 


3 


.244 


22 


.414 


3 


13 


.455 


27 


.432 


4 


1 


.734 


11 


.486 


5 


2 


.312 


4 


.354 


6 


2 


.458 


2 


.458 


*For code 


used. 


see Table 


8 (page 


56) . 



phase, it was observed that the search procedures seemed to 
be unrelated to sex since there was a wide variation in success 
even though 23 of the 28 students were women. A similar 
absence of a specific pattern relating approach to successful 
searches was noted. 

In regard to work experience and formal instruction in 



I the use of libraries, it seemed most likely that the under- 

graduates selected would not have had much of either. In 
fact, of 94 Illinois participants, 60 had neither instruc- 
tion nor work experience; 27 of 50 Wisconsin participants in- 
dicated neither work experience nor instruction. Table 20 
is a summary of the responses to questions 12 and 13 of the 
General Information Form. 



The amount of instruction in "how to use the library" 



must be considered minimal at best. At Illinois, ten students 
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TABLE 20 

SUMMARY OF INSTRUCTION AND WORK EXPERIENCE 
AMONG BOTH GROUPS OF PARTICIPANTS 



Library Experience 


Illinois 


Wisconsin 


Neither instruction nor 
work experience 


60 


(63.8%) 


27 


(54%) 


Instruction only 


20 


(21.3%) 


15 


(30%) 


Work experience only 


6 


(6.4%) 


6 


(12%) 


Both instruction and 
work experience 


8 


(8 . 5%) 


2 


(4%) 



recorded that they had received such instruction as part of a 
general orientation program. Twenty-two students indicated 
that they had completed a series of library assignments as 
part of a course given in the Division of General Studies. 
Only one student indicated he had received instruction in a 
special course, part of the required work in a course in 
historiography. The total number of responses, 33, repre- 
sents 28 students, some of whom had indicated more than one 
item. At Wisconsin, 13 responses were indicated under the 
category of "instruction as part of a general university 
orientation course," and four responses as part of the re- 
quired assignment in a regular course (Freshman English) . 

The amount of work experience was also nominal. Most 
students indicated that their work experience consisted of 




general circulation and shelving assistance over a period of 
one year or less. The types of libraries represented in the 
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Illinois responses were: high school (eight students), 

academic (three) , public (one) and elementary school li- 
braries (three) . Work in high school libraries was indicated 
by five Wisconsin students, three had experience in elementary 

i 

school libraries and one in an academic library. Only one 
student at each school had worked in more than one type of 
library. 

The three variables, sex, work experience, and instruc- 
tion, lend themselves to natural dichotomies ("none" or "some" 
used for the latter two) • The dichotomous measures can be 
used to compute another type of product moment correlation 
known as the point biserial correlation coefficient • 

The calculated value in respect to mean success score for each 
characteristic is given in Table 21. 

TABLE 21 



POINT BISERIAL CORRELATION (rp^) FOR 
SEX, INSTRUCTION AND WORK EXPERIENCE 



Characteristic 


Students 

Wisconsin Illinois 


Sex 


.1629 


.1033 


Instruction 


.0036 


.0004 


Work Experience 


.0758 


.1119 



Each value was tested for significance using a variant 
form of the t-test for difference between means and none was 
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found to be significant. The experimental study data sup- 
port the original assumption, therefore, that these measures 
need not be considered in the matching procedure. 

Students were also requested to indicate the approach 
they most frequently used to search for material in the 
catalog. Each student indicated one of the following: 1) most 

frequently look for material by author or title; 2) most 
frequently look for material by topic (subject) ; or, 3) use 
the subject approach about as often as the author- title 
approach. Students were grouped by approach and a mean of the 
success scores was computed for each group. The means, shown 
in Table 22, were tested for difference by analysis of variance. 

TABLE 22 

GROUP MEANS BY APPROACH 



Illinois Wisconsin 



Approach 


N 


Mean 


N 


Mean 


Author -Title 


26 


.416 


24 


.433 


Subject 


38 


.429 


11 


.415 


Both 


30 


.432 


15 


.409 



The F-statistic for the Wisconsin group, F = .052, and 
the value for the Illinois group, F = 0.070, were not sig- 
nificant. 

Conclusions . — The analysis of the relationship between 
personal characteristics and mean success score for the study 
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giToups indicatod no sig^nif icant association sxcspt £o]T gDrads 
point avsrags • On tlis other hand, there was no evidence that 
the matching procedure followed was in any way invalid. 

Given a large sample from each institution, in fact, the 
simple procedure of random pairing without attention to these 
characteristics would have been appropriate. 



The Validity of the Test Instrument 
The question of validity inevitably arises when dealing 
with any instrument of measurement. The committees on test 
standards of the American Educational Research Association 
and the National Council on Measurements Used in Education 
have identified four types of validity. 

Content validity is concerned with the sampling 
of a Specified universe of content. 

Concurrent validity is concerned with the rela- 
tion of test scores to an accepted contemporary cri- 
terion of performance on the variable which the test 
is intended to measure. 

Predictive validity is concerned with the rela- 
tion of test scores to measures on a criterion based 
on performance at some later time. 

Construct validity . More indirect validating 
procedures, which we refer to under the name con— 
struct validation, are invoked when the preceding 
three methods are insufficient to indicate the 
degree to which the test measures what it is intended 
to measure.^ 



^American Educational Research Association and National 
Council on Measurements Used in Education, Technical Recom - 
mendations for Achievement Tests (Washington: National Edu- 

cation Association^ 1955) , p. 16. 
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The limitations of the present study, as well as the 

absence of a body of pertinent data from previous research, 

precludes the approach of concurrent or predictive validation. 

In addition, since what is measured is not a construct, the 

0 

latter validation procedure is not applicable. Some con- 
sideration, therefore, must be given to content validity. 

The procedure for determining content validity is to 
compare the content of the test with the content of the uni- 
verse to be measured. Within the objectives of the study and 
the nature of the hypothesis to be tested, the test instru- 
ment was developed to measure the ability of students to 
differentiate between subject and non-subject entries. The 
universe, therefore, consists of all the possible entries to 
be found in the catalogs being tested. 

The degree to which the test represented the universe 
was affected by the considerations given to two other problems 
First, it was deemed unreasonable to expect students to give 
more than one hour of their time to the study, thus limiting 
the number of problems that could be included. In addition, 
the variety of forms of subject headings was too great to 
allow representation of each type and required grouping them 
into more general categories. Hence, instead of considering 

^Claire Selltiz, £t Research Methods in Social 

Relations (Rev. one— vol. ed.; New York: Holt, Rinehart 

and Winston, 1959) , p. 41, 
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some twelve forms as categorized by Eaton, ^ the more general 
forms "personal name entries," "corporate name entries, 
"criticisms" and "conventional subject headings" were used. 

The procedure described in Chapter II, including random 
selection of the initial group, was undertaken to ensure 
maximum content validity. Despite these efforts, and a 
limited degree of substantiation from observations of the 
responses of the participants, the assumption of validity 
relies heavily upon the judgment of the investigator. 

One effort to determine the fairness, if not the 
validity, of the test exercises was undertaken. While the 
mean score of each question was expected to vary from school 
to school, it seemed reasonable to expect that the relative 
difficulty encountered in the eight problems would be the same 
for each group of students. Therefore, it was predicted that 
if each question was ordered by degree of difficulty as repre- 
sented by the mean scores, the rank order would be the same 
for the Wisconsin group as for the Illinois group. Table 23 
contains the mean score for each question as computed and the 
rank, in order of difficulty, for each group of students. 

A Spearman— rho rank order correlation coefficient 
was computed. The calculated value, r^ = .922, was 

^Thelma Eaton, Cataloging and Class if icaticpn; — An 
troductory Manual (3rd ed.. Champaign, • Distribute 

by the Illini Union Bookstore, 1963), p. 131. 

^^Sidney Siegel, Nonparametric Statistics for the Be^- 
havioral Sciences (New York: McGraw-Hill Book Company, Inc., 

1956), 202-13. 



o 



I 



87 



TABLE 23 

MEAN AND RANK ORDER OF THE EIGHT TEST QUESTIONS 



Question 


Wisconsin 
Mean Rank 


Illinois 
Mean Rank 


Chemistry as a Profession 


.020 


1 


.117 


1 


Statistical Design 


.327 


2 


.356 


3 


Identification of Firearms 


.640 


7 


.684 


6.5 


Amateur Athletic Union 


.510 


5 


.543 


5 


Great Britain Board of Trade 


.400 


4 


.128 


2 


Hebbel's "Agnes Bernauer" 


.367 


3 


.370 


4 


Otto Ludwig 


.624 


6 


.684 


6.5 


Jackson Pollock 


.788 


8 


.861 


8 



significant at the .05 level. It was concluded that each 
question represented the same degree of difficulty, in com- 
parison with the test as a whole, and was not biased in favor 
of one group or the other. 

Furthermore, an analysis of the means within each group 
reveals that the differences between means is significant. 

This would support the original contention that search re- 
sults are affected by the type of question — and the corre- 
sponding part of the catalog — being used. As additional 
support that the arrangement of the catalog did not affect 
the results of searching for subject references, the means for 
each question were compared between schools. Since it was 
suspected that the variances would be different, the jt— test 
for difference between means without pooled variances was 
used.^^ The calculated values are given in Table 24. 

^^Edwards, o£. cit . , pp- 273-74. 

ERIC 

-ili’lliiHilfflffTlTlMJ ^ — . .. 
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TABLE 24 

TEST FOR SIGNIFICANT DIFFERENCE 
BETWEEN MEANS FOR EACH QUESTION 



Mean Success Scores Calculated 



Question 


Wisconsin 


Illinois 


;t-statistic 


Chemistry as a Profession 


.020 


.117 


2.506* 


Statistical Design 


.327 


.356 


0.355 


Identification of Firearms 


.640 


.684 


0.550 


Amateur Athletic Union 


.510 


.543 


0.386 


Great Britain Board of Trade .400 


.128 


3.607* 


Hebbel's "Agnes Bernauer" 


.367 


.370 


0.037 


Otto Ludwig 


.624 


.684 


1.045 


Jackson Pollock 


.788 


.861 


1.201 


*Significant at the . 


05 level. 






An approximation of 


the expected 


ib-statistic was made 


and it was determined that 


for N = 50 


(Wisconsin) and N = 94 


(Illinois) , t would be expected to have a value 


greater than 



lo986 and less than 2.008, depending on the variances for 
each set of means. Table 23 shows that for six of the eight 
questions arrangement does not, indeed, affect the resulting 
searches. The difference evident for the other two problems, 
creates some question about general applicability of this 
conclusion. Further analysis of the question-catalog inter- 
action, therefore, was necessary and is described in the 
following chapter. 



Summary 

Mean success scores were calculated for the 31 pairs 
of students and used to test the hypothesis that dividing the 
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catalog would increase effective use. From the evidence of 
the experimental groups, there was no reason to reject the 
null hypothesis. Assuming the validity of the appropriate- 
ness of the matching procedure, the divided catalog did not 
result in a more effective catalog for subject searches. An 
examination of the frequencies of successful location of 
cards for known- item searches also indicated no differences 
between the two groups. Therefore, it was determined that 
the arrangement of the catalog did not affect the results of 
subject or known- item searches. 

The analysis of the relationship between personal 
characteristics and mean success scores for the study groups 
indicated no association except for grade point average 
(Illinois group only). On the other hand, there was no evi- 
dence that the matching procedure followed was in any way 
invalid. 

A comparison of the responses to the eight-question 
test instrument indicated that the test represented similar 
problems and relative difficulty for group. The mean 

success scores for each question among each group of students, 
in fact, indicated that there is a greater difference attrib- 
utable to the question than to the arrangement of the catalog. 
An analysis of this interaction between difficulty and type 
of question seemed warranted and is undertaken in Chapter V . 



CEIAPTER V 



DESCRIPTIVE ANALYSIS OF SEARCH PROCEDURES 

In the preceding chapter, the analysis of the data was 
undertaken to test the hypothesis and to investigate the 
validity of some underlying assumptions. The results of this 
analysis indicated that arrangement did not significantly 
affect the successful retrieval of subject references. In ad- 
dition, there was no direct evidence that the test results 
were related to any of a variety of participant attributes, 
except for grade point average for the Illinois group. 

The testing procedure, however, indicated that the 
degree to which participants were successful in coping with 
the search requests varied from problem to problem. This 
difficulty could not be associated with the differences in 
the arrangement of the catalog alcne, nor was the varying 
difficulty due to differences in the personal characteristics 
of the participants. The difficulties participants had in 
using the catalog might only be explained by an interaction 
of the question and the corresponding complexity of the 
catalog encountered. It is the purpose of this chapter to 
describe some observable search procedures. In doing so, it 
is the intention to show the causes of difficulty associated 
with the various problems as well as to indicate some general 
procedures common to many users. 
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Causes of Failure 



During the data collection process, it was observed that 
a number of factors could be considered as the possible cause 
of failure or partial success. Four observable reasons were: 

1) failure to select the appropriate search term, 2) the 
patron's inability to cope with alphabetizing (i.e., filing) 
regulations, 3) small peculiarities in the individual drawers 
at the two schools deemed unimportant in the original matching 
procedure, and 4) the inability of the patron to distinguish 
subject entries from non-subject entries. These four may not 
be the only sources of difficulty in using catalogs, but they 
do constitute general categories represented by observed pro- 
cedures and patterns of searching. In those cases when the 
reason for failure was not clear, a fifth category was used 
in which the cause was indicated as "other reasons." 

There were, however, certain decisions that had to be 
made in the assignment of the cause of failure in order to be 
consistent and to allow some comparability. For example, stu- 
dents searching for subject entries under the phrase heading 
"Chemistry as a Profession" frequently searched under the 
heading "Chemistry, Profession," "Chemistry— Profession" or 
"Profession (s)—Chemistry." This procedure was evident to 
the investigator but it was not inconceivable that the patron 
really began to search for "Chemistry as a Profession" and, 
having difficulty with the filing rules (subdivisions before 
phrases), ended up searching the subdivisions. 
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The question was whether this constituted selection of 
the wrong search term or difficulty with alphabetizing? Since 
such procedures were common to both groups, and since the 
search under the wrong term was obvious and the problem with 
the filing is speculative, the reason for failure was attri- 
buted to "selection of the wrong search term." While this 
may be considered a subjective judgment on the part of the 
investigator, every effort was made to remain consistent in 
the assignment of reasons for failure for all students. 

A second problem involved the decision of what con- 
stituted the major, or primary, cause of failure. Illinois 
students who were searching for "Chemistry as a Profession," 
for example, frequently began looking under various sub- 
divisions of "Chemistry." Frustrated in their efforts to 
locate an appropriate heading, many turned to searching under 
"Profession (s) . " Quite often, students came across two title 
added entries which seemed appropriate: "The Profession of 

Chemistry." A few students went to the effort to inform the 
investigator that these were subject references even though 
such amplification had not been solicited. The decision, in 
this case, was that the first cause of difficulty was with 
the selection of the appropriate search term rather than con- 
fusion about what constituted a subject reference and non- 
subject added entry. 



The small peculiarities cited as one reason for failure 
were not considered important in the original matching of 
sections of the catalog. The introduction of filing flags 
(see Figure 2) at Wisconsin presented some difficulty to stu- 
dents. At Illinois, the use of temporary cards which were 
larger than the catalog cards — thus protruding above the 
regular cards — also caused some problems. Students (in the 
post-test interview) explained that these cards were some 
type of guide card which indicated the end of an appropriate 
section and the beginning of another. 



FIGURE 2. FILING FLAG (Superimposed Over a 
Catalog Card) USED IN THE WISCONSIN CATALOG. 




It was hypothesized that the reason division would pro- 
duce more effective results in conducting a search for subject 
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references was that the divided catalog made subject entries 
explicit. In the dictionary catalog, on the other hand, 
participants could be observed selecting title added entries 
and other non-subject added entries in the belief that these 
were subject references. The division of the catalog, how- 
ever, does not completely alleviate this difficulty. It was 
assumed that there would be individuals who did not realize 
that the catalog was divided. For those who didn't, this 
represents as much of a misunderstanding about the differ- 
ences between subjects and non-subject references as that 
faced by the user of the dictionary catalog. For the pur- 
poses of comparison, "Tisconsin participants who used the 
author-title catalog for subject searches were equated to 
those who selected non— subject entries at Illinois. 

In regard to this point, three interesting phenomena 
occurred at Wisconsin. First, some participants were com- 
pletely unaware of the division of the catalog and had no 
success in locating any subject references. A second group 
of students "discovered" the subject catalog in the process 
of the exercises. For them, the test instrument was in- 
structive. A third group seemed to move freely from one 
catalog to the other without ever realizing the difference. 
The investigator could only conclude that this was a matter 
of accident, depending on what part of the alphabet they 
encountered. Students would select a search term, walk down 
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the center aisle separating the catalogs and then find the 
correct part of the alphabet. Whether this was in the 
author-title or subject catalog seemed to make no difference. 

Tabulation of the causes of failure . — All responses 
were reexamined and a cause of failure assigned for each 
question. The causes were then coded as given in Table 25 

TABLE 25 

CODE USED FOR SUCCESS AND CAUSES 
OP FAILURE AND PARTIAL SUCCESS 

Code Explanation 

Sq Success; no difficulty. 

Pjl^ Selection of wrong search term. 

P 2 Difficulty with filing rules. 

P^ Peculiarities in the catalog. 

P^ Confused by non-subject entry. 

P^ Other; variables with question. 



and tabulated for each question. The results of this tabula- 
tion are given in Table 26 and 27. 

Inspection of the causes of failure or partial success 
indicate differences between schools greater than might be 
expected inasmuch as the mean scores for each question are so 
similar. Since Table 26 includes data for all participants 
it is possible that the differences noted might be due to dif- 
ferences in the personal attributes of the two groups. When 
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TABLE 26 

FREQUENCY OF SUCCESS AND CAUSES FOR FAILURE OR PARTIAL 
SUCCESS BY QUESTION FOR ALL WISCONSIN (W) AND 

ILLINOIS (I) STUDENTS 



Question 


Stu- 

dents 


^0 , 


F 

1 


F 

2 


F 

3 


F 

^4 


^5 


Chemistry as a 


(W) 


1 


45 


0 


0 


4 


0 


Profession 


(I) 


11 


82 


1 


0 


0 


0 


Identification of 


(w) 


16 


23 


2 


3 


6 


0 


Firearms 


(I) 


33 


14 


47 


0 


0 


0 


Statistical 


(W) 


32 


10 


0 


0 


6 


2 


Design 


(I) 


26 


3 


1 


0 


40 


24 


Amateur Athletic 


(W) 


25 


19 


3 


0 


3 


0 


Union 


(I) 


48 


26 


0 


0 


18 


2 


Gt. Brit. Board of 


(W) 


20 


15 


10 


0 


5 


0 


Trade 


(I) 


10 


38 


23 


3 


20 


0 


Hebbel ' s "Agnes 


(W) 


18 


8 


0 


0 


22 


2 


Bernauer" 


(I) 


26 


51 


0 


0 


17 


0 


Otto Ludwig 


(W) 


2 


0 


0 


10 


38 


0 


(I) 


12 


0 


0 


1 


81 


0 


Jackson Pollock 


(w) 


38 


0 


3 


0 


7 


2 




(I) 


61 


0 


0 


0 


0 


33 


Total: 


(w) 


152 


120 


18 


13 


91 


6 




(I) 


227 


214 


72 


4 


176 


59 
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TABLE 27 

PERCENT OF SUCCESS AND CAUSES FOR FAILURE OR PARTIAL 
SUCCESS BY QUESTION FOR ALL WISCONSIN (W) AND 

ILLINOIS (I) STUDENTS* 



Question 


Stu- 

dents 


So 




^2 


^3 


^4 


^5 


Chemistry as a 


(W) 


2.0 


90.0 


0.0 


0.0 


8.0 


0.0 


Profession 


(I) 


11.7 


87.2 


1.1 


0.0 


0.0 


0.0 


Identification of 


(W) 


32.0 


46.0 


4.0 


6.0 


12.0 


0.0 


Firearms 


(I) 


35.1 


14.9 


50.0 


0.0 


0.0 


0.0 


Statistical 


(W) 


64.0 


20.0 


0.0 


0.0 


12.0 


4.0 


Design 


(I) 


27.7 


3.2 


1.1 


0.0 


42.5 


25.5 


Amateur Athletic 


(W) 


50.0 


38.0 


6.0 


0.0 


6.0 


0.0 


Union 


(I) 


51.1 


27.6 


0.0 


0.0 


19.2 


2.1 


Gt. Brit. Board of 


(W) 


40.0 


30.0 


20.0 


0.0 


10.0 


0.0 


Trade 


(I) 


10.6 


40.5 


24.5 


3.2 


21.2 


0.0 


Hebbel's "Agnes 


(W) 


36.0 


16.0 


0.0 


0.0 


44.0 


4.0 


Bernauer " 


(I) 


27.6 


54.3 


0.0 


0.0 


18.1 


0.0 


Otto Ludwig 


(W) 


4.0 


0.0 


0.0 


20.0 


76.0 


0.0 


(I) 


12.8 


0.0 


0.0 


1.1 


86.2 


0.0 


Jackson Pollock 


(W) 


76.0 


0.0 


6.0 


0.0 


14.0 


4.0 




(I) 


65.0 


0.0 


0.0 


0.0 


0.0 


35.0 


Total: 


(W) 


38.0 


30.0 


4.5 


3.2 


22.8 


1.5 




(I) 


30.2 


28.3 


9.6 


0.5 


23.4 


7.8 



*The percentage shown is the frequency in each cell 
divided by the total number of searches for each row. The 
sum of each row is 100 percent (variation is due to rounding) • 



98 



only the 31 matched pairs are considered, however, similar 
discrepancies were evident. Therefore, a closer analysis of 
the search patterns and difficulties encountered is warranted. 

Analysis of the Cause of Failure 

Peculiarities in each catalog (F 3 ) . — Of the five cate- 
gories enumerated as causes of failure, the small peculiari- 
ties within each catalog accounted for the smallest number 
of failures or partial successes. Of the 17 cases in which 
filing flags or temporary catalog cards presented some diffi- 
culty, over one-half of these problems were associated with 
the Wisconsin students' searches for material about (rather 
than by) Otto Ludwig. 

File flags were located in two places in the Ludwig 
file which had a direct effect on the search for appropriate 

^ One flag was located between the fifth and sixth 
entry and the second was found between the last and second 
last cards. Four students indicated the file ended at the 
first flag. Some students searched the entire file including 
the criticisms of Ludwig's works. Among this group, a few 
recognized the difference between biographical and critical 
works and, in returning to the appropriate file, inadvertantly 
neglected to include the last card. Thus six participants 
indicated 28 of 29 appropriate cards, kept from a perfect re- 
sponse by the appearance of a filing flag. In general, how- 
ever, such difficulties constituted slightly more than 2 percent 
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of all failures or partial successes and appear insignificant 
in comparison to other difficulties. At the same time, it 
illustrates that those catalog characteristics which had 
appeared inconsequential did prove to be a source of diffi- 
culty to a small number of patrons. 

Ot her problems (F 5 ) . — Some problems, noted as ' 3lso 

were minor except in two cases. There were a few students 
who searched under such headings as "Ameteur , " "He^el &.nd 
"Habbel" even though the correct spelling was provided. More 
important, however, was the consistent difficulty that 
occurred in respect to the questions about Jackson Pollock 
and "Statistical Design." 

The Pollock files were identical in size (three cards) 
in type (all subject added entries) . For some reason, 33 
Illinois students (and two at Wisconsin) selected only two 
of the three cards as appropriate. The only possible dif- 
ference that could be observed in all cases was that the 
selected cards had personal names as the main entry while 
the third card was a corporate main entry. Some supporting 
0 vidence that this may, indeed, have been the reason for 
such a decision can be advanced on the basis of observed 
procedures in connection with other questions. 

Many students, for example, could not conceive of 
corporate bodies as "authors" and this was exemplified by 
the difficulty students had with the questions dealing with 
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corporate entries as subjects. In many cases, the corporate 
main entry (in the Pollock file) was identified as the 
"publisher." In the search for other biographical material 
(Ludwig) , students also identified the appropriate cards by 
the fact that a second personal name appeared under the name 
of the person about whom they were seeking information. The 
fact that the searches were made in the subject catalog at 
Wisconsin, or contained "red headings" at Illinois, was not 
sufficient to overcome some notions individuals had about 
corporate entries. 

The search for material on "Statistical Design" provided 
evidence of the varied meaning associated with cross refer- 
ences. Students who searched for the unused term "Statistical 
Design" found a cross reference to the used term "Experimental 
Design." This "see" reference was interpreted as having the 
same meaning as a "see also" reference by 24 Illinois stu- 
dents and two Wisconsin students. Thus, the Illinois stu- 
dents selected the two title added entries appearing after 
the cross reference as appropriate and preferred to consider 
the entries under "Experimental Design" (which were not 
searched) as a last alternative. The two Wisconsin students 
decided that nothing on the subject was available. 

Perhaps more enlightening was the decision of many of 
the students that they would (and did) search the "Experi- 
mental Design" file but that they would normally — that is 
when conducting a search for one's own need — select only 
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those titles that expressly mentioned "statistics" and would 
exclude the others. There was little question that these 
students (at Illinois and Wisconsin) did not consider the two 
terms as synonymous but rather as a reference from a specific 
term to a more general, inclusive term. 

Difficulty with filing rules (F 2 ) . — Two of the eight 
questions revealed that the responses were affected by various 
complexities of the filing rules used in library catalogs. 

A comparison, between schools, of the responses to the 
"Identification of Firearms" question, indicated a great 
discrepancy in the number of successful searches. This was 
directly attributable to difficulty with the alphabetical 
arrangement. Both catalogs treat the term "Firearms" as a 
single word and file such entries after headings in which 
"fire" is the first of one or more terms (e.g.. Fire, Fire 
alarms. Fire-worshipers, etc.). 

The basic difficulty was that students did not realize 
that they were searching under the term as if it were two 
words "Fire Arms." Most students continued to search through 
the file and at Wisconsin they frequently discovered the 
correct entry (but not, necessarily, the correct subdivision) . 
At Illinois those who continued scanning the file encountered 
a reference card as shown in Figure 3. To all but three of 
the students this cross reference was meaningless. Indeed, 
of the two Illinois students who searched under the exact 
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FIGURE 3. "FIREARMS" REFERENCE CARD 
LOCATED IN THE ILLINOIS CATALOG 



Fire-arms industry and trade. 

See cards filed: Firearms industry and 

trade. 




heading "Identification of Firearms" and found the cross 
reference to "Firearms — Identification," one had already en- 
countered the "Fire-arms" cross reference and decided it 
would be futile to return to the "Firearms" drawer. (The 
second student began in the "Identification" section and had 
no difficulty in reaching the appropriate section.) 

The second question which was affected by the filing 
rules was the search for material about (but not by) the 
Great Britain Board pf Trade. Twenty percent of the Wisconsin 
students had difficulty in coping with the rules for filing 
as compared to 23 percent of the Illinois students. Unlike 
the "Firearms" search, the difference in the successful 
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location of the appropriate cards (40 percent at Wisconsin 
and 11 percent at Illinois) would seem to be due to other 
factors. It was interesting to note the difference in the 
way students approached the problem in comparison to the 
U.S. Civil Service Commission — not included in the final 
analysis. 

As noted in Chapter III (pages 60 and 61 ) , the number 
of students who had difficulty in coping with the alphabetical 
arrangement at Illinois and Wisconsin varied in respect to 
these two questions. It seems plausible that one reason is 
that students, from previous experience, have become con- 
ditioned to the apparent difficulty of using the "U.S." 
files but do not have a similar reservation about headings 
for other countries — perhaps expecting the files to be smaller. 
Again, the nature of the test instrument prevented gathering 
conclusive evidence for this hypothesis, but a comparison 
of the three corporate entries reveals that the rate of 
success was inverse to the file size and that the single 
most important factor associated with failure was selection 
of the wrong search term. 

The use of incorrect search terms (Fi) . — The largest 
single cause associated with complete failure in locating 
appropriate cards for any request was the selection of in- 
correct search terms. In an attempt to minimize this diffi- 
culty, the questions submitted to the students were in terns 



that appeared in the catalog. In some cases, these were 
unused terms for which a cross reference was provided. 
Nevertheless, of the 1,152 searches included in this analysis, 
334 (29 percent) were conducted using the wrong term or 
terms . 

For most questions, the headings selected were more 
general than the request itself. In addition, many of the 
searches were conducted using key concepts contained in the 
request, frequently inverted. For the corporate entries, the 
most frequently used headings were "Civil Service — U.S.," 
"Boards of Trade — Great Britain," and "Athletics." The most 
popular (but incorrect) terms used in connection with con- 
ventional subject headings were "Chemistry — Profession, " 
"Profession — Chemistry," "Design — Statistical," and "Firearms." 

In contrast to this difficulty in selecting the appro- 
priate term, searches for material about individuals pre- 
sented no problem in either catalog. More serious diffi- 
culty was associated with the request for material about 
Hebbel's "Agnes Bernauer." A fairly large number of Illinois 
students (54 percent) actually chose to look for criticisms 
under the title of the work "Agnes Bernauer" or in the more 
conventional form (for a person) under "Bernauer, Agnes." 

A much smaller number of Wisconsin students (16 percent) used 
these approaches . 
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Associated with the search procedure using incorrect 
terms for corporate entries was an unusual pattern in making 
the final decision of which cards w€ire pertinent to the re- 
quest. A number of students indicated that the appropriate 
cards were contained within the general files located (for 
example, "Civil Service — U.S.") without further explanation. 
Among those students who did not select the complete file 
as appropriate, there were many who indicated entries for 
which the main entry was the corporate body given in the re- 
quest. Some effort was made, during the post-test interview, 
to ascertain the logic behind this latter procedure. Some 
students responded that material "published" by the agency 
should contain some information about the agency. From 
others, there were vague expressions that the cards selected 
seemed to be the most logical ones in the file. 

Inability to distinguish between subject and non- 
subject entries (F 4 ) . — The major hypothesis of this study 

was directly concerned with this category of difficulty. The 
fact that 23 percent of the students had difficulty in ^ 
distinguishing between the two types of entries makes this 
the second largest contributor to failure or partial success 
in making subject searches. Yet for three questions, 
"Chemistry as a Profession," "Identification of Firearms," 
and "Jackson Pollock" this difficulty was observed only 
among the users of the divided catalog. In addition, the 
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use of the inappropriate author-title catalog in the search 
for criticisms occurred more frequently than the selection 
of non-subject entries by the users of the dictionary catalog. 
On the other hand, confusing non-subject entries for subject 
references was more common among the users of the dictionary 
catalog in coping with the other four questions than among 
the divided catalog users. The chief example of the nature 
of this problem was seen in connection with the searches for 
material about ‘'Statistical Design." 

A number of Illinois participants (42 percent) included 
title added entries as well as subject entries for the "Sta- 
tistical Design" question. This did not include the 24 per- 
cent who had selected only the two titles filed after the 
cross reference referring to the used term "Experimental 
Design." The ten titles most frequently cited were considered 
to be — by the participant — more specific subdivisions of the 
general subject being searched. 

Not all the confusion, however, was centered around 
that of subject versus non-subject entries. For the Ludwig 
search, students had difficulty in distinguishing between 
subject added entries ,(i.e., biographical material) and author- 
title subject added entries (i.e., criticisms). While this 
may seem a highly technical point, the difficulty students 
had with the Hebbel criticism would indicate that there is 
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a minimum of understanding of the fine distinction between 
the two types of entries. 

The implications of these observations, not only in 
respect to the research design, but to improving the use of 
library catalogs, is discussed as part of the general summary 
and conclusions in Chapter VI. 

Summary 

The purpose of this chapter was to investigate the 
nature of the relationship between the various questions and 
the difficulties encountered in each catalog. The results of 
this analysis can be summarized in respect to two general 
elements of the study. 

First, the question-catalog interaction, which was 
assumed and was the basis for constructing a multi-exercise 
test instrument, was substantiated. Second, the general con- 
clusion that arrangement does not significantly affect the 
successful retrieval of subject references is tenable. It 
was observed that for any possible benefit attributable to 
divided catalogs (for example, a larger percent of successful 
searches for "Statistical Design") there may be a correspond- 
ing disadvantage (re; "Chemistry as a Profession"). 

These conclusions, however, are appropriate only to the 
two groups participating in the study and for the specific 
questions used. Generalization to undergraduates in large uni- 
versities and to all types of subject searches can be made only 
if the samples used in this study can be deemed representative. 



CHAPTER VI 



SUMMARY AND CONCLUSIONS 

ThQ purpose of this study was to investigate the pro- 
posal, set forth by a number of librarians at various times 
during the past thirty years, that dividing a library catalog 
would result in improved effective use of the catalog. The 
design of the study undertaken to test this hypothesis re- 
quired the resolution of a number of problems. 

Review of the Procedure 

The study was developed in terms of the variables perti- 
nent to a catalog search— the question, the catalog and the 
user. To test the hypothesis, the same search should be made 
by the same patron in catalogs alike in all respects but 
arrangement. Practical limitations necessitated certain 
changes in this theoretical design. 

A set of search-questions was developed as the data 
collection instrument. The questions were obtained by random 
selection of personal and corporate entries from the diction- 
ary catalog that had been selected for use in the study. In 
addition, a number of conventional subject headings were 
selected randomly from a standard list of subject heading^. 
The final test instrument also included two known-item search 
requests, 
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Two catalogs — one in dictionary arrangement (Illinois) , 
the other divided into subject and non-subject entries 
(Wisconsin) — were selected and the appropriate sections 
verified in regard to similarity in size and complexity. 

The testing was conducted during the spring semester of 
1966-67 with undergraduates from the two universities whose 

catalogs had been chosen. 

EXoinsnts o£ tlis 

The causal element . — A number of reasons have been 
enumerated to explain the difficulties encountered by cata- 
log users. These reasons include the selection of the wrong 
search term, problems with the filing of catalog entries, and 
failure to understand or interpret correctly the meaning of 
"see also" references. None of these difficulties, however, 
can be alleviated by dividing the catalog. In fact, during 
this study all of these causes of failure were observed among 
the users of both types of catalogs. In addition, both 
groups of students demonstrated that as much confusion exists 
about the meaning of "see" references as had been observed 
about "see also" references in previous studies. 

These studies have also indicated that confusion of 
title entries for subject headings by patrons caused diffi- 
culties in the use of the catalog. The divided catalog, by 
providing a separate file for subject entries, makes the sub- 
ject approach more explicit. It was believed, therefore. 
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that the number of non-subject references that would be in- 
cluded in subject searches among catalog users would be 
decreased among the divided catalog users thereby producing 
an increase in the effective use of the catalog. 

Defining effective use . --Measuring effective use was 
one of the important questions that had to be considered. A 
review of previous catalog use studies revealed not only a 
series of shortcomings in design for testing the effective- 
ness of the catalog, but a relatively weak definition of 
effective use. The method of measuring effectiveness — and 
thereby inferring a definition of effective use — was essen- 
tially that of determining how frequently the patron was 
satisfied in his search for some material. This definition 
requires the acceptance of two underlying assumptions of 
questionable validity. First, it is assumed that the results 
of a single subject search represent the average ability of 
an individual. A series of such searches was then used to 
infer the effectiveness of the catalog. This does not, how- 
ever, guarantee that all types of questions have been asked 
nor does it account for the possible difficulty that one 
patron may have when searching for another subject and 
encountering a more complex part of the catalog. 

For this study, each participant was requested to com- 
plete a series of catalog searches. As a result of this 
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procedure, it was clearly demonstrated that the nature of 
the question and the complexity of the catalog were as 
important in determining effective use as was the ability 
of the participant. 

A second assumption of the definition of effective- 
ness in previous studies was that the material located was 
more relevant to the participant's need than some other 
reference. The concept of relevancy is not a simple problem 
to resolve© What may be a relevant document for one indi- 
vidual may be irrelevant for a second, even though they both 
may have selected the same index term to make the search. 

The process of deciding what document is relevant to a need 
can only be made by the user. The user, however, cannot 
exercise his maximum option of choice unless he has located 
all pertinent references. 

It was upon this concept that the decision was made to 
test the students' ability to locate pertinent references 
but not to ask for a judgment of the appropriateness of the 

document . 

Scoring the results . — Given the definition of effective 
use— i.e., the location of all pertinent subject references— 
a measure designated the "mean success score" was developed. 
This measure provided for crediting the searcher for locating 
pertinent references and penalizing him for including inappro- 
priate references. The meaningfulness of this score, for 
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testing tlie hypothesis f was based on an important assump- 
tion. It was assumed that any differences in the scores 
would be due to the hypothesized causal element (i.e., con- 
fusing non-subject entries for subject entries) and that 
other reasons for failure would be relatively consistent. 

The analysis descicibed in Chapter V, however, pro- 
vided some evidence that cast doubt on the validity of 
this assumption. It was found that some differences in the 
mean success score were due to types of difficulties other 
than the inability to distinguish subject entries from non- 
subject entries. A question-by-question analysis illustrated 
that the causes of failure were not consistent for each ques- 
tion either between catalogs or for any one single catalog. 
Therefore, while the mean success score does indicate how 
well an individual has performed — in terms of the definition 
of effective use — it does not tell what was the cause of 
difficulty. 

The matching procedure . — Another assumption, upon which 
the validity of the investigative procedure was based, con- 
cerned the appropriateness of the two criteria selected for 
the initial matching of the respondents. An analysis of 
these two criteria — exposure and familiarity — as v/ell as 
other characteristics of the participants was undertaken. 

Data on the following attributes were collected: 1/ semesters 

on campus and class standing, as measures of exposure to the 
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catalog; 2) frequency of use of the main catalog as a 
measure of familiarity; 3) sex; 4) cumulative grade point 
average; 5) most common approach to using the catalog; 

6) the type and amount of instruction received in "how to 
use the library"; and 7) work experience in libraries. 

The analysis of the relationship between personal 
characteristics and mean success score for the study groups 
indicated the absence of any association except between 
grade point average and mean success score. Based on this 
series of analyses it was determined that the matching pro- 
cedure followed was appropriate. This conclusion, however, 
is not intended to imply that these characteristics are not 
related to the ability to use catalogs effectively. For a 
relatively homogeneous group, given a similar educational 

level such as university undergraduates — the range of 

differences in the various characteristics considered is so 
narrow that matching on additional criteria would seem 
unwarranted • 



General Findings 

The specific hypothesis tested was: 

Assuming all other factors are equal, subject 
searches through a catalog in which the subject 
entries have been separated (o.g*, 3 divided cata- 
log) will produce more pertinent references and 
fewer inappropriate references than identical 
searches using a file combining all entries into 
a single (dictionary) sequence. 
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Effective use of the catalog was measured in terms of 
a mean success score for subject searches. This score 
represented the ability of students to select appropriate 
subject references in response to a series of questions. 

The experimentally derived data v;ere tested for signifi- 
cance using a test for difference between means. The 
means were found not to be different and it was concluded 
that dividing the catalog was not a satisfactory device for 
making subject searches more effective. 

An analysis of the difficulties students had in coping 
with various questions was undertaken to determine if such 
difficulties were associated with arrangement. The results 
of this analysis indicated that for any potential benefits 
attributable to the divided catalog (i.e., a larger percentage 
of successful searches for one or more questions) there were 
corresponding disadvantages (i.e., lower rates of success for 
other questions) . Furthermore, the percent of failures 
attributed to the causal element was almost as great for the 
users of the divided catalog (22,8 percent) as for the users 
of the dictionary catalog (23.4 percent). It was concluded, 
therefore, that for the two groups in general, the divided 
catalog did not facilitate subject searches more than the 
dictionary catalog. 

The effect of arrangement on known-item searches was 
also investigated. Students were requested to determine if 
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•fheir rsspsctiv© library containGd two spscific docurnsrits. 

An analysis of the responses indicated that the rate of 
success or failure in obtaining the call nuinber for the two 
documents was not related to the differences in the arrange- 
ment of the two catalogs. 

Conclusion . — As a result of this study, it was con- 
cluded that for a series of questions representing different 
levels of difficulty and for a fairly limited range of cata- 
log use habits and ability, changes in the arrangement of 
large catalogs would not result in more effective use. At 
the same time there was no overwhelming evidence obtained in 
this study to indicate that one type of catalog was distinctly 
superior to the other. 

Implication . — Subject to the limitations described be- 
low, this study presents some data to assist the academic li- 
brarian in making a decision about the arrangement of the 
catalog. The decision may be made in terms of cost of pro- 
duction and maintenance with some degree of confidence that 
the choice of arrangement will not affect the use of the 
catalog by the undergraduates of the institution. 

Lmitations of the Findings 

Every study has inherent limits — both conceptual and 

practical which define the degree of generalization that is 

possible. In the interest of maintaining maximum control 
over the various elements of catalog searches, the choice of 




participants, catalogs, and questions was highly structured. 

j[jQposes a fairly severe limitation to the genexalization 
of the findings to non-participants. On the other hand, some 
inferences to other groups may be made, given the acceptability 
of some assumptions. 

Previous research has indicated that subject searches 
constitute approximately half of the use made of catalogs. 

Since it was contemplated that changes in arrangement would 
not affect the results of known-item searches — substantiated 
in this study — it was natural to emphasize the subject ap- 
proach. Another generalization from previous studies can 
also be considered. 

^]^ere is some evidence to indicate that educational 

2 

level is associated with successful use of catalogs. Diffi- 
culties encountered by undergraduates might be assumed to 
be common to users with less educational preparation. No 
inferences can be made to those who represent a higher level 
on the educational continuum. There is some evidence, how- 
ever, to indicate that many individuals who are representa- 
tive of this upper level (i.e., graduate students) do not 
use the catalog for subject searches as frequently as for 

^Carlyle Jo Prarey, "Studies of Use of the Subject 
Catalog: Summary and Evaluation," in Maurice P. Tauber, ed., 
The Subject Analysis of Library Materials (New York: School 

of Library Service, Columbia University, 1953) , p. 162. 

^Ibid., p. 155. 
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known-item searches. Therefore, exclusion of this latter 
group is not as serious as it may appear at first. 

No single study can provide conclusive evidence in sup- 
port of a single hypothesis. The conclusions reached in 
this study are subject to a type II error (that is, a true 
hypothesis has been rejected) . One way to decrease the 
probability of making such an error, and thereby increase 
the power of the test, is to increase the size of the sample. 

In addition, even if the groups selected for participation in 
the study are truly representative of academic library users 
(which is not contended) , there is little basis for making 
inferences to groups with other characteristics. It may be, 
for example, that for large urban public libraries with 
their variety of users, dividing the catalog would increase 
effective use. 

Suggestions for Further Research 

At least three different approaches can be explored 
further in an attempt to improve catalog use. First, more 
and better instruction of all potential users of the catalog 
might be undertaken. Second, continued efforts in the improve- 
ment of the catalog through changes in form, format and 
arrangement might be investigated. Third, a radical departure 
from the traditional concept of a public catalog and more 
direct professional service to patrons may be attempted. 

3 

Margaret Cornelia Brown, "The Use of the Subject Cata- 
log by Graduate Students in the Social Sciences." (Unpublished 
Master's thesis. Graduate Library School, University of 
Chicago, 1946), p. 53. 
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Improved instruction . — It was noted previously (Chapter 1/ 
p. 11) that giving more and better instruction to catalog 
users may be a difficult, perhaps unrealistic goal* There 
is, at the present, no conclusive evidence that would help 
to establish the appropriate level of knowledge and 
familiarity that such instruction must attempt to reach. It 
may be fruitful to evaluate the increase in effective use 
among those who have had a one semester course in "how to 
use the library." Representatives from such a group were 
not included in the Illinois sample used in this study 
although a one semester course is offered to undergraduates 
by the Graduate School of Library Science. The evidence 
gathered in this study, however, indicates that a few hours 
of formal or informal instruction does not have an apparent 
influence on the ability of undergraduates to use the cata- 
log. These findings cannot be considered conclusive because 
of the sampling procedure used, and are also subject to 
verification . 

Improving the catalog . — Improvement in the catalog it- 
self would appear to be a more fruitful approach to increas- 
ing effective use. It was within this context that the 
present study was undertaken. While the crisis-like atmos- 
phere associated with the information "explosion" and the 
need for bibliographic control has given impetus to studies 
dealing with the problems of bibliographic entry and subject 



analysis, there are indications that failure to consider 
the problems of form, format, and arrangement may have 
serious consequences • 

Writing on the economics of book catalog production, 
Hayes, Shoffner, and Weber indicate that judgments to be 
made in selecting the form and content of a book catalog are 
hindered by the paucity of serious studies in this area: 

The human engineering of the book catalog 
divides into three major aspects, each of which 
deserves further study. One is the design of 
the bibliographical entry, treatment of cross 
references, formatting of the secondary entry, 
and syndetic relationship to supplements and 
indexes. The second is the formatting for ease 
of scanning. ... The third aspect is the 
packaging for ease in handling . . .'^ 

In contrast, Lancaster and Mills, in writing about the 
Cranfield Project, dismiss the effect of display (i.e., file 
organization) as an important factor in determining operating 

5 

efficiency of a retrieval system. The limitation of this 
conclusion, as Swanson has pointed out, is that the indexers 
also conducted the searches and the indexer's memory may have 
been the reason for results that showed no difference in re- 
trieval between the four systems of indexing tested.^ Even 



"^Robert M. Hayes, Ralph M. Shoffner, and David C. Weber, 
"The Economics of Book Catalog Production," Library Resources 
and Technical Services , 10 (Winter, 1966), 65-66. 

F. W. Lancaster and J. Mills, "Testing Indexes and 
Index Language Devices: The ASLIB Cranfield Project," 

American Documentation , 15 (January 1964) , 4-13. 

Don R. Swanson, "The Evidence Underlying the Cranfield 
Results," Library Quarterly , 35 (January 1965), 11. 
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if memory is not an important factor, it is important to 
point out that the individucils had expertise in the in- 
tricacies of bibliographic organization, an assumption that 
cannot be inferred to a more general population represented 
by the lay catalog user. 

There is an almost inexhaustible number of changes and 
improvements that may be proposed for the catalog. For 
example, with the proliferation of book catalogs in recent 
years, it is important to determine whether book catalogs 
actually are an improvement in terms of effective use or 
whether their advantages and disadvantages are limited to the 
economics of production and maintenance. 

Perhaps a more promising development, which has impli- 
cations for all types of catalogs regardless of form, would 
be the establishment of classified catalogs as replacements 
to the more traditional dictionary (and divided) catalogs. 

n 

McGeever's study' provided some information about the users 
of one specific classified catalog, but little is known about 
the effectiveness of the classified catalog. 

The traditional unit card, as exemplified by printed 
Library of Congress cards, is also subject to criticism. 

There is, certainly, a need to determine if more information 
might not be conveyed (and more accurately) if changes in the 
formatting of the unit card were made. There are, of course, 

^Emmett Bernard McGeever, "A Study of Use of a Classi- 
fied Catalog." (Unpublished Master's thesis. Graduate Library 
School, University of Chicago, 1958) , pp. 2-3. 



a number of changes that have been suggested which range from 
photoduplication of the title page, advocated almost 90 years 
ago,® to Voos' more recent proposal of a vertically-divided 
card.® 



Finally, numerous studies have indicated that patrons 

do not completely understand the meaning of various types 

of cross references r Mostecky has suggested a rather elabo- 

« 

rate revision of such cards to indicate the relationship of 
various subject headings to more specific terms and to 
broader terms as well as to other related subjects.^® In 
contrast, the simple process of changing the words "see" and 
"see also" to more precise terminology (such as the "refer to 
used in soitie thesauri of des'criptors) may be sufficient. 

While these two recommendations may represent ^the eJ^remes in 
suggested changes, the obvious need for improvement of cross 
references in library catalogs is well documented. 

Professional servicing of the subject catalog . — Perhaps 
the third suggestion for improvement in the use of catalogs 
represents the furthest departure from traditional concepts. 

®Henry Stevens, "Photo-bibliography; Or, A Central 
Bibliographical Clearing-House," Edward B. Nicholson and 
Henry R. Tedder, eds.. Transactions and Proceedings of the 
Conference of Librarian~Held in London October, 1877 (London 
Printed at the Chiswick Press by Charles Whittingham, 1878) , 
pp. 70-80. 

®Henry Voos, "Revision of the Current Library of Con- 
gress Card Format," Library Resources and Technical Services , 
11 (Spring 1967), 167-72. 

^®Vaclav Mostecky, "Study of the See-Also Reference 
Structure in Relation to the Subject of International Law," 
Am.erican Documentation, 7 (Octol.er 1956) , 294-314. 
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The catalog is not a simple bibliographical too3- to under- 
stand and use^ The ability to recognize subject headings, 
for example, is a fairly elementary concept. Yet for many 
users this may represent a rather sophisticated idea. If 
this is true there is a wealth of information the lay user 
does not use, e.g., tracings, notes on cards, the classifi- 
cation number and cross references to related subjects » 

If it is acceptable to provide professional assistance 
to users of other types of bibliographies and indexes (e.g., 
periodical indexes, special subject bibliographies, etc.) it 
would seem logical to include the subject analysis of the li- 
brary collection which the subject catalog represents within 
this area of responsibility. The proposal, then, is to 
make the subject portion of the catalog a part of the reference- 
bibliography service of the library and not to treat it as a 
separate entity in itself. As with other reference tools, 
the individual who begins to feel competent in the use of 
the subject index through continual assistance will also 
begin to make more and more independent searches. At the 
same time, the availability of professional assistance will 
be explicitly clear and not implied as is so often the 
present case in many libraries. 

The proliferation of separate undergraduate libraries 
on major university campuses would seem to afford an ideal 
situation to test the practical application of this approach. 
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The suggestion' that subject catalogs be dispensed with com- 
pletely may seem to be a drastic solution. On the other hand; 
a continuation of the policy of expecting the lay catalog 
user to become competent enough to use the catalog at its 
maximum potential may be as drastic, and unrealistic, as 
well • 

In the development of the problem dealt with in this 
study the original thesis was that professional librarians 
had one of two choices for increasing the usefulness of the 
library catalog. The first was to provide more and better 
instruction for users of library catalogs. In addition to 
the limitations of such an approach, which were discussed 
earlier, it seems clear that education of patrons requires 
dealing with an element of the problem over which the pro- 
fessional librarian has little control. The alternative 
was to consider what efforts might be undertaken to improve 
the product itself (the catalog) and thereby indirectly 
influencing the patron's ability to use the catalog. While 
this second approach remains tenable and subject to further 
study, it is apparent that the amount of change that must 
take place may be very extensive. 

As a result of this study, which afforded the oppor- 
tunity to observe closely almost 200 students in a relatively 
short period of time, another concept was developed. The 
individual student's approach was mentally compared with 
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what might be expected of librarians attempting the same 

search. Through a process, yet unclear, of educational 

preparation and experience, a librarian develops the ability 

% 

to approach a problem by cons ■'dering and using a wide range 
of alternatives. These include the choices between dif- 
ferent entries and forms of subject headings (filing dif- 
ferences) , alternatives suggested by tracings, etc. Stu- 
dents in this study, while exhibiting some flexibility, per- 
formed each catalog search in a fairly narrow, and frequently 
predictable, fashion. The approach suggested — professional ' 
servicing of the catalog — would seem to offer greater 
possibilities for securing information about the user and 
the adequacy of the catalog than any other method of research 
yet undertaken. 
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APPENDIX A 

FORMS USED IN PRE-TEST PHASE 

LIBRARY RESEARCH CENTER 
428 LIBRARY 



L.S. 255 CATALOG USE EXERCISE 



PURPOSE 

The intent of this study is to secure "normal" scores 
for an exercise which is being developed to measure the 
effective use of a large university library catalog. The 
results of this exercise will in no way affect your grades 
in the course nor will there be a similar exercise given at 
the end of the semester. The general results, however, will 
be used to determine whether certain areas of the L.S. 255 
curriculum may need further emphasis during the remainder of 
the semester as well as for future L.S. 255 classes. 



PROCEDURE 

1. Sign up for a one-hour period on the sheets posted 
on the bulletin board located outside of Room 314. 
Please allow a full hour for this exercise; do not 
schedule your hour immediately before another class. 

2. Report to Room 422 at the time you have selected 
to participate. The room is located in the library 

on the 4j^ floor just above Room 314 and can be reached 
most easily by using the North stairway. 

3. You will be asked to complete a brief "Informa- 
tion Sheet." All information provided will be kept 
confidential and will be used only to make some 
statistical tests. 

4. The catalog use exercise will be completed at the 
main catalog of the University library (2nd floor) in 
the presence of the assistant. 

James Kr ike las 
Research Associate 
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APPENDIX A (contd,) 

LIBRARY RESEARCH CENTER 
UNIVERSITY OF ILLINOIS 

GENERAL INFORMATION SHEET 
(Conf ident ial ) 

SEARCH NO, 

I. PERSONAL AND EDUCATIONAL INFORMATION 

Sex: ( ) Female ( ) Male 

Age: 

Official Class designation: 

( ) FresLman ( ) Junior ( ) Master' s Program 

( ) Sophomore ( ) Senior ( ) Doctoral Program 

Is this your first semester in attendance at this 
university: ( ) No ( ) Yes 

Undergraduate subject field: 

Major: — 

Minor (if any) : 

II. LIBRARY EXPERIENCE 

A. How frequently have you used the main public card 
catalog in the central library? 

( ) Less than once a semester 

( ) 1 or 2 times a semester 

( ) 3 or 5 times a semester 

( ) 6 to 10 times a semester 

( ) 11 to 15 times a semester 

( ) 16 or more times a semester 
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APPENDIX A (contd.) 

II. LIBRARY EXPERIENCE (contd.) 

B. In the use of the card catalog do you 

( ) most frequently look for material by author or title? 

( ) most frequently look for material by topic (subject)? 

( ) use the subject approach about as often as the 

author-title approach. 

C. Have you ever received instruction in "hovv to use the 

library" in one or more of the followings 

1. as part of a general university orientation course? 

( ) No 

( ) Yes. If yes, at what institution? 

2. as part of one of your regular classes? 

( ) No 

( ) Yes. If yes, in which course? 

3. in a special library-use course? 

( ) No 

( ) Yes. If yes, what was the name of the course? 

. Name of the institu- 
tion where taken: • 

4. informally from a library staff member? ( ) No ( ) Yes 

D. Have you ever worked in a library? 

( ) No 

( ) Yes. Was its ( ) a high school library? 

( ) a college or university library? 

( ) a public library? 

( ) other (specify) s . 

What work did you perform? Please be specific (e.g., 
shelving, repairing books, circulation assistant. 

How long have you worked in libraries? 
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APPENDIX B 

FINAL SEARCH-PROBLEMS PRESENTED TO THE PARTICIPANTS 



Known-Item Request 



Does the library have, and if so what is the call number of 
the "Warren Report" submitted by the President's Commission 
on the Assassination of President John F. Kennedy? 



Does the library have, and if so what is the call number of, 
the book "The Structure of Science" by Ernest Nagel? 



Conventional Subject Requests 

Locate the appropriate catalog cards that indicate the library 
contains material about the topic: 

Chemistry as a profession 



Locate the appropriate catalog cards that indicate the library 
contains material about the topic: 

Identification of firearms 



Locate the appropriate catalog cards that indicate the library 
contains material about the topic: 

Statistical design 



Criticism Request 

Assume you have the text of the title listed below, locate the 
appropriate catalog cards that indicate the library con- 
tains material about: 

The tragedy by Friedrich Hebbel entitled "Agnes Bernauer" 
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APPENDIX B (contd . ) 

Individual-As-A-Subiect Request 

Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

Otto Ludwig (a 19th Century writer) 



Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

Jackson Pollock 



Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

Honore Gabriel Riquetti (Count Mirabeau) 



Corporate Entrv-As-A-Subject Request 

Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

The Amateur Athletic Union of the UoS. 



Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

The Great Britain Board of Trade 



Locate the appropriate catalog cards that indicate the library 
contains material about (rather than by) : 

The U.S. Civil Service Coinmission 
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APPENDIX C 

FORMS USED FOR FINAL TEST GROUPS 



LIBRARY RESEARCH CENTER 
428 LIBRARY 



27 January 1967 



The Library Research Center of the University of Illinois is 

undertaking a study, supported by a 

mine how to make library catalogs more 

students. We have received permission to solicit 

of students at the University of Wisconsin and your name has 

appeared in a sample drawn from the student population. 

Each participant will be requested to conduct a few short 

searches using the card catalog. „ -.ata 

less than one hour and will be conducted 
loq of the University's main library. In addition, yo 
will be asked to complete a brief "information sheet. 

All information will be treated confidentially and at no 
time will the published results of this study identify 
specific individuals. Each student will receive payment of 
$1.50 for his participation in the study. 

The success of any project such as this is 
voluntary cooperation of participants. By the nature 
the sampling method used, your participation is representa- 
tive of approximately one hundred fifty of your fellow 
students. One of our objectives is to identify catalog- 
use habits of a wide variety of students, 

who have had little occasion to use the university library. 
May we express our sincere hope that you will find time to 
participate in this study. 

Will you please complete the enclosed postcard reply and 
return no later than February 4, 1967? You will be 
contacted by the 8th to set up the exact time for the 
exercise. Every effort will be made to schedule you during 

the week of February 6-11, 1967. 



Sincerely yours. 



James Krikelas 
Research Associate 



JK/lc 
end . 



er|c 




appendix C (contdo) 






^ THIS SIDE OF CARD IS FOR ADDRESS } 

Professor John J • Boll 
Library School of the 

University of Wisconsin 
425 Henry Mall 
Madison, Wisconsin 5370b 

ATTN: James Kr ike las 



.J 



r”' 



(PLEASE COMPLETE AND RETU^ BY FEBRUARY 4, 1967) 

Yes, I will participate in the catalog-use study 
being conducted. 



My campus phone number is: — 

The best time to contact me is (give appropriate 
days and hours; e.g., Tuesday & Thursday, 

4-6 p.m.) : 



Signature 







ittiaii 
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appendix C (contd.) 



LIBRZyiY RESEARCH CENTER 
428 LIBRARY 



3 February 1967 



The Library Research Center is undertaking a study, sup^rted 
by a federal grant, to determine how to make library cat^ogs 
more useful to university students. We have received pern s- 
sion to solicit the help of students at the University of 
Illinois and your name has appeared in a sample drawn from 
the student population. 

Each participant will be requested to complete a few short 
searches using the card catalog. 

less than one hour and will be conducted at the main catalog 
of the University's main library. In addition, we are en- 
closing an "information sheet" which is to be completed and 
returned prior to participating in the search exercise. All 
information will be treated confidentially and is intende 
only for the purpose of statistical analysis; at no time 
will the published results of this study identify specific 
individualso Each student will receive payment of $1.50 for 
their participation in the study. 

The success of any project such as this is dependent on the 
voluntary cooperation of participants. By the nature of the 
sampling method used, your participation is representative 
of apprLimately one hundred fifty of your fellow students. 
One of our objectives is to identify catalog-use habits of 
a wide variety of students including those who have had 
little occasion to use the university library. May we 
exprpss our sincere hope that you will find time to partici- 
pate in this study. 

Will you please complete the enclosed "information sheet" 
and return it not liter than February 15, 1967? You will 
be contacted by the 20th to set up the exact time for the 
exercise. Every effort will be made to schedule you during 
the week of February 20-25, 1967. 



C -i nr'<a-rol \r 






James Krikelas 
Research Associate 

JK/lc 

end. 



o 
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APPENDIX C (contd.) 



LIBRZ^Y RESEARCH CENTER 
428 LIBRARY 



GENERAL INFORMATION FORM 
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APPENDIX C (contd.) 



B. CATALOG USE (continued) 

10. How frequently have you used one or more of the 

departmental library catalogs? 

[ ] Less than once a semester 

[ ] 1 or 2 times a semester 

[ ] 3 to 5 times a semester 

[ ] 6 to 10 times a semester 

[ ] 11 to 15 times a semester 

[ ] 16 or more times a semester 

11. In the use of any card catalog do you 

[ ] most frequently look for material by author or 

title? 

[ ] most frequently look for material by topic 

(subject) ? 

[ ] use the subject approach about as often as the 

author-title approach? 

C . LIBRARY EXPERIENCE 

12. Have you ever received instruction in "how to use the 

library" in one or more of the following: 

a) as part of a general university orientation course? 

[ ] No [ ] Yes 

b) as part of one of your regular classes? 

[ ] No [ ] Yes 

c) in a special bibliography or library science course? 

[ ] NO [ ] Yes 

13 . Have you ever worked in a library? 

[ ] No 

[ ] Yes. If yes, was it: [ ] a high school library? 

[ ] a college or univer- 

sity library? 

[ ] a public library? 

[ ] other (specify) : 
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APPENDIX D 

INVESTIGATOR'S INSTRUCTIONS TO EACH PARTICIPANT 

The purpose of this study is to determine how , rather 
than why, you use the library's main catalog. To achieve this 
objective we have prepared a set of twelve problems which I 
am going to ask you to answer by using the main catalog only. 
Before explaining the procedure, I'd like you to complete 
this General Information Form. [AT ILLINOIS: request stu- 

dents to clarify any answer that is in question.] The infor- 
mation you give is to be used for general statistical analysis 
and at no time will you or your responses be identified by name. 

The procedure will be to go to the catalog and to search 
for answers to each question. I am interested in two basic 
types of information. One is the procedure [search pattern] 
you use to seek the answers. The second, is your final de- 
cision about which cards are appropriate. In regard to this 
second point, you may arrive at one of two decisions: that 

there are no cards appropriate to the request in the main 
catalog; or, you may find one or more cards which are appro- 
priate. If your decision is the latter, indicate exactly 
what you have located by showing me the card(s) and, at the 
same time, tell me in your own words what your decision is. 
Inasmuch as I will be observing your procedure it is not 
necessary to write your response on the cards (although you 
may if you wish) thus saving some time. 
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APPENDIX D (contd.) 



The questions have been chosen randomly and it is con- 
ceivable that they represent requests about which you have 
no knowledge or interest. It is assumed, however, that your 
basic ability to use the catalog should allow you to reach 
some decision. 

If you arrive at a point where you cannot think of any- 
where else to look or when you would ordinarily seek help 
from the library staff, just Indicate that to me,, we'll con- 
sider that particular search at an end and you may proceed 
to the next problem. 

ARE THERE ANY QUESTIONS? Let me caution you to read 
each problem carefully and to be sure in your mind that you 
understand what you are being asked for. Do not assume all 
requests are identical. You may begin. 






i 
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APPENDIX E 

CALCULATIONS FOR t-TEST 
FOR DIFFERENCE BETWEEN MEANS 
(MATCHED OBSERVATIONS) 



Summary from data in Table 13: 
N = 31 



= 15.518 



ZX2 = 13.328 



Zx. 



^1 = 



15.518 _ e/M V _ 

—JJ— - .501 X2 - 



Zx, 



13.328 



N 



N 

Sd = 2.190 

Standard error of the difference for matched groups: 



31 



Ld = 1.820 



^ = 1.820 - = 1.665 



N 



31 



c,2 _ 


1.665 


= .0555 


®d “ N-1 


30 




y/.0555 


= 0.236 




.236 


■ = Q 0423 


Cf 

to 

II 


yii 





t-Test for significance: 



t = 



^1 - ^2 
Sxi-X2 



.501 - .430 
.0423 



= 1.678 



.430 
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APPENDIX F 

STATISTICAL FORMULAE 



[NOTE: The computation of the success ratio was carried 

out to three places after the decimal even if such mathe- 
matical precision was not significant. This was done to pre- 
vent misinterpretation due to rounding a number such as 0.996 
to 1.0. In addition, some computations resulted in interven- 
ing zeros before the significant figure and would have re- 
quired the presentation of data in an awkward and uneven 
manner (e.g., numbers of varying length such as 0.98, 0.031, 
0.0056, etc.). The calculated success ratio and mean success 
scores were entered on machine-readable cards for computer 
processing. The computer output was recorded in eight digits 
and the appropriate decimal designation. These figures were 
also rounded to three digits after the decimal. 

The various correlation coefficients (Pearson— r) re- 
ported in Chapter IV were done by computer. Other outputs 
were requested for the variables needed for each formulae 
and the corresponding statistic calculated by the investi- 
gator. These formulae are given in a form most convenient 
for computation and may represent slight variations from the 
formulae given in the sources cited in the text.] 



Correlation Coefficient (Pearson-r) 



Formula: 



[ ZXY - 



r = 



1 



[ 



N 



Zx Sy , 

N 

[ 2 ■ ] 



Where: X = semesters in college, or semesters on 

campus, or grade point average; 

Y = mean success score; 

N = number of students for each group. 



appendix P (contd.) 
Analysis of Variance 
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Formulae : 



Sum of squares (total) = SS^ = 



( SX) 

N 



Sum 



of squares (between) = SSj^ 



SSn 



= ( 



( EX2) 



n. 



n. 



■4* • • • 






n . 

1 



( Sx) 



‘1 2 
Sum of squares (within) = SS„ = SS^ - SSj^. 

Mean square = MS = SS/df j (df = degrees of freedom) . 

F = MSjj/MS^. 

VJhere: X = Mean success score; 

N = Totai number of students ; 

n , n , etc. = Number of students in each group, 



Example of computer produced values; 




Example of calculation of F-statistic: 



Source 


SS 


df 


MS 


F 


Between 


.068 


3 


.023 


0.92 


Within 


2.016 


82 


.025 


Total 


2.084 


85 





"'''•‘•“‘"'•'•"‘'"'"Hill 
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APPENDIX F (contd.) 
Point Biserial Correlation 



Formulae: 



r 




Yq - npZYt 

J n^rij^[n^ZLY^ - ( EY^) ] 



t = ^pb J M - 2 

1 

Where: = Mean success score for male students (or 

® students with no instruction, or no 

work experience) ; 

= Mean success score for female students 
^ (or students with "some" instruction, 

or "some" work experience) ; 

Y = Mean success score for all students; 



“^0 


= Number 


of 


"o 


students ; 




= Number 


of 


II V >> 
^1 


students ; 


"t 


= Number 


of 


II V II 

^t 


students • 



Spearman-rho Rank Order Correlation (r^) 



Formulae : 




6 Ild^ 

- N ■ 



N = number of elements ranked. 
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APPENDIX F (contd.) 

t-T est for Difference Between Means 
wi-hhout Pooled Variances 



Formulae : 



S 



2 

1 




(£x)^ 





r 
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(1)_ (2) (3) (4) (5) (6) (7) (8) (9) (10 ) (11) OH 



022 


1 


3 


5 


5 


1 


3.3 


3 


0 


0 


.478 


.425 


023 


1 


3 


5 


3 


2 


4.1 


1 


0 


0 


.598 


.532 


025 


1 


1 


1 


1 


1 


4.8 


1 


1 


0 


.473 


.420 


026 


1 


1 


1 


1 


2 


3.1 


1 


0 


1 


.556 


.495 


027 


1 


3 


5 


1 


2 


3.4 


3 


0 


0 


.223 


.198 


029 


2 


3 


5 


1 


1 


3.9 


1 


0 


0 


.473 


.420 


030 


2 


4 


7 


7 


4 


4.9 


3 


0 


0 


.728 


.647 


031 


2 


4 


7 


7 


6 


3.8 


3 


1 


0 


.598 


.643 


032 


1 


1 


1 


1 


1 


3.8 


3 


0 


0 


.598 


.532 


034 


1 


1 


1 


1 


1 


3.9 


3 


0 


0 


. 125 


. Ill 


035 


2 


3 


5 


5 


1 


3.6 


1 


0 


0 


.723 


.035 


036 


1 


1 


1 


1 


1 


3.6 


3 


1 


0 


.371 


.330 


037 


2 


4 


7 


7 


1 


3.6 


1 


1 


0 


.603 


. 536 


038 


2 


3 


5 


5 


3 


3.6 


2 


0 


0 


.348 


.313 


039 


1 


4 


7 


6 


4 


3.1 


2 


1 


0 


. 996 


. 885 


041 


1 


3 


5 


5 


4 


4.0 


1 


0 


0 


. 130 


. 115 


042 


1 


1 


1 


1 


1 


3.7 


1 


0 


1 


.598 


. 532 


044 


1 


1 


1 


1 


3 


3.3 


3 


0 


1 


. 125 


. Ill 


045 


1 


1 


1 


1 


2 


3.6 


1 


1 


0 


.598 


. 532 


046 


2 


3 


4 


4 


4 


3.8 


1 


0 


0 


. 125 


. Ill 


047 


1 


2 


3 


1 


4 


4.6 


2 


1 


1 


. 683 


. 718 


048 


2 


1 


1 


1 


1 


3.6 


2 


0 


0 


. 723 


. 643 


049 


1 


3 


5 


1 


2 


3.3 


3 


0 


0 


.496 


.441 


050 


1 


4 


7 


7 


5 


3.0 


2 


0 


0 


.223 


.198 


052 


2 


3 


5 


1 


1 


3.0 


2 


1 


0 


.000 


.000 
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053 


2 


4 


9 


9 


3 


3.5 


1 


1 


0 


. 542 


.482 


054 


2 


3 


5 


6 


6 


3.3 


3 


0 


0 


.848 


. 865 


055 


2 


1 


1 


1 


1 


2.0 


3 


0 


0 


.250 


.222 


056 


1 


1 


2 


3 


1 


3.7 


1 


0 


1 


.598 


.532 


057 


1 


3 


5 


5 


1 


3.4 


1 


0 


0 


.746 


.663 


058 


1 


3 


5 


1 


3 


4.2 


1 


1 


0 


.000 


.000 


059 


2 


2 


3 


2 


1 


3.9 


1 


0 


0 


.638 


.567 


060 


2 


4 


7 


7 


1 


3.5 


1 


0 


0 


.228 


.202 


451 


1 


2 


1 


1 


3 


4.7 


3 


0 


0 


.612 


.550 


453 


1 


1 


1 


1 


3 


3.9 


2 


1 


0 


.651 


.579 


455 


2 


2 


3 


3 


2 


3.6 


2 


0 


0 


.389 


.346 


456 


1 


2 


3 


3 


2 


3.7 


1 


0 


0 


.435 


.401 


457 


2 


1 


1 


1 


1 


3.7 


2 


0 


0 


.442 


.419 


458 


2 


1 


1 


1 


1 


4.6 


3 


0 


0 


.478 


.425 


459 


1 


4 


7 


7 


4 


3.4 


1 


0 


0 


.250 


. 222 


460 


1 


4 


7 


3 


3 


4.2 


3 


0 


0 


.346 


.313 


461 


2 


2 


3 


3 


2 


3.9 


3 


0 


0 


.298 


.315 


462 


2 


1 


1 


1 


1 


2.5 


3 


1 


0 


. 267 


. 238 


463 


2 


3 


5 


5 


3 


4.3 


2 


0 


0 


. 192 


. 171 


465 


1 


1 


1 


1 


6 


4.4 


2 


1 


1 


. 725 


.644 


466 


1 


2 


3 


3 


2 


4.0 


2 


1 


0 


.401 


.356 


467 


2 


1 


1 


1 


2 


4.6 


2 


0 


0 


. 234 


. 208 


468 


1 


2 


3 


3 


4 


4.6 


2 


0 


0 


. 755 


. 719 


469 


1 


2 


3 


1 


1 


3.9 


3 


0 


0 


. 442 


. 393 


471 


2 


4 


7 


7 


4 


4.5 


2 


0 


0 


. 580 


. 516 
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( 1 ) 


( 2 ) 


( 3 ) 




( 5 ) 




( 7 ) 


( S ) 


(91 


( 10 )- 


( 11 ) 


( 1 - 2 ) 


472 


1 


2 


3 


3 


3 


3.3 


3 


0 


1 


.434 


.386 


473 


2 


2 


3 


3 


4 


3.3 


2 


1 


0 


.667 


.593 


474 


2 


2 


3 


3 


3 


3.1 


1 


0 


0 


.281 


.250 


475 


1 


1 


1 


1 


5 


3.8 


2 


0 


0 


.548 


.537 


476 


2 


3 


5 


2 


2 


3.6 


1 


1 


0 


.381 


.339 


477 


2 


3 


5 


5 


3 


3.2 


3 


0 


0 


.899 


.799 


478 


2 


2 


3 


3 


1 


3.1 


2 


0 


0 


.453 


.403 


479 


2 


1 


1 


1 


3 


3.9 


1 


1 


0 


.342 


.329 


480 


1 


3 


5 


3 


2 


3.2 


3 


0 


0 


.466 


.414 


482 


1 


4 


7 


7 


1 


4.5 


3 


0 


0 


.856 


.872 


483 


1 


3 


5 


5 


5 


3.3 


2 


0 


0 


.347 


.364 


484 


1 


4 


7 


7 


2 


3.6 


1 


0 


0 


.213 


.190 


485 


2 


3 


5 


5 


3 


3.4 


3 


0 


0 


.483 


.410 


486 


1 


2 


3 


3 


4 


3.6 


2 


1 


0 


.417 


.370 


487 


2 


1 


1 


1 


1 


3.0 


1 


0 


0 


.315 


.280 


488 


1 


2 


3 


3 


1 


2.8 


2 


0 


0 


.266 


.237 


489 


1 


1 


1 


1 


3 


4.0 


3 


0 


0 


.429 


.381 


490 


1 


1 


1 


1 


3 


3.0 


1 


1 


1 


.362 


.321 


491 


2 


3 


5 


2 


3 


3.7 


2 


0 


0 


.430 


.382 


493 


1 


3 


5 


1 


3 


4.2 


1 


1 


0 


.529 


.470 


494 


2 


2 


3 


3 


3 


3.6 


2 


0 


0 


.481 


.428 


495 


1 


4 


7 


7 


1 


3.4 


3 


0 


0 


.371 


.330 


497 


2 


4 


9 


5 


1 


3.0 


2 


0 


0 


.347 


.308 


498 


1 


3 


5 


3 


4 


3.6 


2 


0 


0 


.347 


.308 


499 


2 


4 


7 


7 


3 


4.0 


1 


0 


0 


.661 


.588 
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( 1 ) ( 2 ) ( 3 ^ ( 4 ) ( 5 ) ( 6 ) ( 7 ) ( 8 ) ( 9 ) ( 10 ) . ( . 1 1 ) ilZl 



500 


2 


2 


3 


3 


2 


3.7 


2 


1 


0 


.396 


.352 


501 


1 


3 


5 


1 


1 


3.5 


3 


1 


0 


.478 


.425 


502 


1 


1 


1 


1 


4 


4.5 


2 


1 


1 


.772 


. .734 


503 


1 


3 


5 


4 


2 


4.0 


1 


0 


0 


.742 


.660 


504 


1 


1 


1 


1 


1 


2.1 


2 


0 


0 


.467 


.492 


505 


1 


4 


6 


6 


4 


3.4 


1 


1 


0 


.474 


.421 


507 


2 


1 


1 


1 


3 


3.8 


3 


0 


0 


.381 


.339 


508 


2 


1 


1 


1 


3 


3.5 


3 


0 


0 


.401 


.356 


509 


2 


4 


9 


9 


1 


4.2 


1 


1 


0 


.481 


.515 


510 


2 


3 


5 


1 


2 


4.0 


1 


0 


0 


.294 


.266 


512 


2 


4 


7 


2 


2 


3.8 


2 


0 


0 


.625 


.451 


513 


2 


4 


7 


7 


2 


3.0 


2 


1 


0 


.500 


.444 


514 


2 


1 


1 


1 


1 


4.3 


1 


0 


0 


.514 


.453 


515 


2 


4 


7 


7 


2 


3.8 


2 


0 


1 


.544 


.515 


516 


2 


1 


1 


1 


1 


3.5 


3 


0 


0 


.314 


.279 


518 


2 


1 


1 


1 


3 


3.1 


2 


1 


0 


.472 


.424 


519 


1 


2 


3 


3 


1 


3.8 


1 


1 


1 


.450 


.400 


520 


2 


2 


3 


3 


2 


3.2 


3 


0 


0 


.442 


.392 


521 


2 


3 


5 


5 


3 


3.7 


1 


0 


0 


.325 


.293 


523 


2 


3 


5 


5 


1 


3.3 


1 


1 


0 


.621 


.552 


524 


2 


1 


1 


1 


6 


4.5 


3 


1 


0 


.305 


.271 


525 


2 


4 


7 


5 


4 


4.7 


2 


0 


0 


.353 


.314 


526 


1 


1 


1 


1 


2 


3.7 


2 


0 


0 


.480 


.426 


527 


2 


1 


1 


1 


1 


4.3 


3 


0 


0 


.578 


.514 


528 


2 


1 


1 


1 


1 


3.7 


3 


1 


0 


.339 


.307 
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( 1 ) ( 2 ) ( 3 ) ( 4 ) ( 5 ) ( 6 ) ( 7 ) ( 8 ) ( 9 ) ( 10 ) ( 11 ) ( 12 ) 



529 


2 


4 


7 


7 


2 


4.5 


3 


0 


0 


.562 


.524 


530 


2 


2 


3 


3 


1 


3.9 


2 


0 


0 


.703 


.625 


531 


1 


3 


5 


5 


5 


3.9 


1 


0 


0 


.480 


.426 


532 


2 


1 


1 


1 


2 


2.3 


2 


0 


1 


.111 


.098 


533 


1 


1 


1 


1 


3 


4.2 


3 


1 


0 


.352 


.313 


534 


2 


3 


5 


3 


1 


3.4 


1 


1 


0 


.394 


.350 


535 


2 


2 


3 


3 


3 


3.7 


2 


0 


0 


.625 


.578 


538 


2 


1 


1 


1 


3 


2.6 


1 


0 


0 


.722 


.753 


541 


2 


4 


7 


5 


2 


4.4 


3 


0 


1 


.722 


.578 


542 


2 


2 


3 


3 


2 


4.5 


1 


0 


0 


.400 


.355 


543 


2 


1 


1 


1 


3 


2.7 


2 


1 


1 


.685 


.609 


544 


1 


1 


1 


1 


3 


4.4 


3 


0 


1 


.452 


.402 


545 


2 


3 


5 


1 


2 


4.2 


3 


1 


1 


.700 


.622 


546 


2 


1 


1 


1 


1 


3.5 


1 


0 


0 


.343 


.305 


547 


2 


1 


1 


1 


1 


3.1 


1 


0 


0 


. 172 


. 153 


548 


2 


1 


1 


1 


1 


3.9 


2 


0 


0 


.338 


.301 


549 


2 


1 


1 


1 


1 


3.1 


3 


0 


0 


.392 


.349 


550 


1 


1 


1 


1 


3 


5.0 


2 


1 


1 


. 500 


.556 


552 


2 


1 


1 


1 


1 


3.8 


3 


0 


0 


.469 


.417 


553 


1 


4 


6 


2 


2 


3.5 


1 


0 


0 


. 990 


.980 


555 


2 


4 


7 


7 


1 


3.4 


1 


0 


0 


.619 


.550 


556 


1 


1 


1 


1 


5 


2.5 


2 


1 


1 


. 098 


. 087 


557 


2 


4 


7 


7 


1 


3.5 


2 


0 


1 


. 728 


.758 


558 


2 


3 


5 


5 


4 


3.7 


3 


0 


0 


.484 


541 

• 


560 


2 


2 


3 


3 


3 


4.3 


2 


1 


0 


^67 


.415 
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( 1 ) 


( 2 ) 


(3) 




( 5 ) 


(6) 


( 7 ) 


J3L 


( 9 ) 


(10) 


(11) 


(12) 


564 


1 


2 


3 


3 


4 


3.2 


3 


0 


0 


.680 


.604 


567 


1 


3 


5 


5 


3 


3.5 


2 


0 


0 


.298 


.265 



^Numbers 005 through 060 were assigned to Wisconsin participants; 
numbers 451 through 567 designate Illinois participants. 



^Class has been coded as follows; 1 = Freshmen, 2 - Sophomores, 

3 = Juniors, and 4 = Seniors. 

3por "Frequency of Use of the Main Catalog" see code as given in 
Table 8 (p. 56). 

^Approach has been coded as follows; 1 = most frequently look for 
material by author or title, 2 = most frequently look for material by 
topic (subject), and 3 ~ use the subject approach about as often as the 
author- title approach. 

^For Instruction and Work Experience, 0 = "none" and 1 = "some." 

^Mean success scores were computed for eight searches for compari- 
son between schools; nine searches were used to computed the mean success 
score for all other statistical analyses (within schools). 
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